mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-10-14 21:40:47 +02:00
Route OpenAI auto removal through Qwen
This commit is contained in:
10 files changed
+67
-46
No files matched your search
@@ -59,10 +59,10 @@ rungs were checked ascending and stopped at the first clean verdict.
|
||||
| Meta | gen_studio_mug 1600x1600 | 0.03 | 0.045 |
|
||||
| Meta | gen_text_poster 1600x1600 | -- | <=0.03 |
|
||||
|
||||
Derived operating points by the shipped rule (worst first-clean boundary plus
|
||||
one observed cross-source spread):
|
||||
Derived operating points from this initial calibration (the OpenAI value was
|
||||
superseded by the 2026-09-07 holdout below):
|
||||
|
||||
- **OpenAI: 0.09** (0.075 + (0.075 - 0.06)). Against qwen-zimage 0.07675 and
|
||||
- **OpenAI: 0.09, superseded.** (0.075 + (0.075 - 0.06)). Against qwen-zimage 0.07675 and
|
||||
sdxl-zimage 0.15. The typography boundary (0.05, 0.06] is seed-stable across
|
||||
seeds 0, 1, 2.
|
||||
- **Meta: 0.17** (0.10 + (0.10 - 0.03)), oracle-verified clean on the worst
|
||||
@@ -284,8 +284,8 @@ Generation script: `scripts/chroma_preship_validation.py`; outputs under
|
||||
Engine quality is cohort-dependent, and after measuring all four cohorts the
|
||||
split is even and sharp:
|
||||
|
||||
- On OpenAI-provenance content Chroma1 is better at every measured fidelity
|
||||
axis at its floor (0.09 vs 0.07675), for comparable money.
|
||||
- On the initial OpenAI fixtures Chroma1 was better at the then-current floor,
|
||||
but the later holdout moved that floor to 0.1375 and changed the router decision.
|
||||
- On Microsoft InvisMark content Chroma1 is better at every measured fidelity
|
||||
axis at a LOWER floor than qwen (0.125 vs 0.15).
|
||||
- On Meta Content Seal Chroma1 needs 1.7x the strength (0.17 vs 0.1) and loses
|
||||
@@ -473,9 +473,29 @@ The committed seed-0 first-clean is not a fluke.
|
||||
2-face extra also clears at 0.06, and only the 9-face grid needs 0.075.
|
||||
That is not a YuNet split: `face_count > 0` would over-strengthen the
|
||||
2-face file, and a high face-count cutoff would be one fixture. The
|
||||
spread (0.075 - 0.06) is already absorbed by the shipped 0.09 floor.
|
||||
spread (0.075 - 0.06) was absorbed by the 0.09 floor for this calibration,
|
||||
but the later holdout below invalidated that operating point.
|
||||
Images API generations remain non-carriers (C2PA only).
|
||||
|
||||
## OpenAI hard-carrier holdout (2026-09-07)
|
||||
|
||||
Two additional withheld OpenAI SynthID carriers expanded the Chroma boundary.
|
||||
Each original was a positive pixel control after metadata-only stripping, so the
|
||||
verdict did not depend on C2PA provenance. Their Chroma first-clean strengths were
|
||||
0.10625 and 0.1375. Both Qwen outputs were clean at the existing 0.07675 operating
|
||||
point.
|
||||
|
||||
The paired fidelity read was mixed rather than a reason to pay Chroma's higher
|
||||
denoise. On the one detected face, Qwen had higher identity cosine (0.835 vs
|
||||
0.821), lower face LPIPS (0.108 vs 0.118), and lower whole-image LPIPS (0.095 vs
|
||||
0.125); Chroma retained more local texture and had slightly higher SSIM. On the
|
||||
second carrier Qwen again had slightly lower LPIPS (0.104 vs 0.109), while Chroma
|
||||
had higher SSIM and PSNR.
|
||||
|
||||
This invalidates both the 0.09 explicit Chroma floor and the OpenAI auto route.
|
||||
Explicit `chroma-zimage` now uses the measured worst-clean rung 0.1375. `auto`
|
||||
uses `qwen-zimage` for OpenAI at 0.07675; Chroma remains the Microsoft route.
|
||||
|
||||
## Content-balanced engine selection check (2026-08-31)
|
||||
|
||||
The earlier tables were intentionally carrier-focused and therefore small. A
|
||||
|
||||
+2
-2
@@ -482,8 +482,8 @@ The same option exists on `all` and `batch`, and as
|
||||
| --- | --- |
|
||||
| `qwen-zimage` | Default. Qwen-Image-2512 global pass plus a SAM-masked Z-Image face stage |
|
||||
| `sdxl-zimage` | The same recipe and face stage on an SDXL global pass, at a higher denoise |
|
||||
| `chroma-zimage` | The same face stage on an Apache-2.0 Chroma1 global pass, with lower OpenAI and Microsoft floors and higher Google and Meta floors |
|
||||
| `auto` | Pick the engine from provenance: chroma-zimage for OpenAI and Microsoft, qwen-zimage for Google, Meta, and unknown |
|
||||
| `chroma-zimage` | The same face stage on an Apache-2.0 Chroma1 global pass; explicit OpenAI use needs a higher floor than qwen-zimage |
|
||||
| `auto` | Pick the engine from provenance: chroma-zimage for Microsoft, qwen-zimage for OpenAI, Google, Meta, and unknown |
|
||||
|
||||
**All four are CUDA-only.** There is no CPU or MPS profile for invisible-watermark
|
||||
removal. The former `controlnet`, `sdxl`, `qwen` and `default` profiles were removed
|
||||
|
||||
@@ -105,7 +105,7 @@ frozen text regions would keep the watermark
|
||||
|
||||
`qwen-zimage` is the default profile. `sdxl-zimage` and `chroma-zimage` keep
|
||||
the same face stage and swap the global model. `auto` picks chroma-zimage for
|
||||
OpenAI and Microsoft provenance and qwen-zimage otherwise. All are CUDA only.
|
||||
Microsoft provenance and qwen-zimage otherwise. All are CUDA only.
|
||||
Qwen and SDXL condition the global stage on a canny edge map, which preserves
|
||||
structure but not identity or exact texture. Chroma1 is a plain strength pass
|
||||
without Canny.
|
||||
@@ -251,8 +251,9 @@ established for that stage.
|
||||
2026-08-29/30 across the four vendor oracles (full record:
|
||||
[`chroma1-engine-research.md`](chroma1-engine-research.md)):
|
||||
|
||||
- OpenAI: `0.09` (below qwen's `0.07675` is false economy neither way -- the
|
||||
matched-strength fidelity favors Chroma1 on this cohort);
|
||||
- OpenAI: `0.1375`. Two withheld carriers expanded the Chroma first-clean
|
||||
boundary beyond the original `0.09` policy while qwen-zimage cleared both at
|
||||
its existing `0.07675` operating point;
|
||||
- Microsoft InvisMark: `0.125` (below qwen's `0.15`);
|
||||
- Google: `0.40` for zero-face content (above qwen's `0.27`; at this floor the
|
||||
regeneration destroys dense text and collapses face identity -- the tradeoff
|
||||
@@ -275,7 +276,7 @@ are certified at a fixed seed. The live resolver is
|
||||
| `qwen-zimage` | CUDA only, large model stack, and limited broad certification across seeds and content. |
|
||||
| `sdxl-zimage` | CUDA only. Its strength ladder is flat per vendor, not a resolution curve, because flat values are what was measured. |
|
||||
| `chroma-zimage` | CUDA only. Higher Google and Meta floors than qwen; Google face content uses 0.125 instead of 0.40. |
|
||||
| `auto` | CUDA only. Routes OpenAI and Microsoft to chroma-zimage and everything else to qwen-zimage. |
|
||||
| `auto` | CUDA only. Routes Microsoft to chroma-zimage and everything else to qwen-zimage. |
|
||||
|
||||
Only manually verified `vae-glyphs` is an optional production stage, and it is
|
||||
experimental rather than a default.
|
||||
|
||||
@@ -1520,8 +1520,8 @@ The current profiles are `qwen-zimage` (the default), `sdxl-zimage`,
|
||||
`qwen` and `default` were removed rather than kept as a CPU path, and are
|
||||
rejected rather than aliased onward. `auto` is a deterministic per-cohort
|
||||
selection policy:
|
||||
chroma-zimage for OpenAI and Microsoft, qwen-zimage for Google, Meta, and
|
||||
unknown. It does not run a learned router or classify the image's genre.
|
||||
chroma-zimage for Microsoft, qwen-zimage for OpenAI, Google, Meta, and unknown.
|
||||
It does not run a learned router or classify the image's genre.
|
||||
|
||||
`qwen-zimage` normally resolves global denoise from image area for unknown content.
|
||||
Measured provider cohorts bypass that curve with flat operating points. The values,
|
||||
@@ -1789,8 +1789,7 @@ Diffusers truncation trap sdxl-zimage documents). There is no Canny
|
||||
conditioning: the floors were measured on a plain strength pass.
|
||||
|
||||
The flat vendor floors (`CHROMA_ZIMAGE_*_STRENGTH` in watermark_profiles):
|
||||
OpenAI 0.09 and Microsoft 0.125 (both BELOW qwen's, with better
|
||||
matched-strength fidelity), Google 0.40 for zero-face content / 0.125 when
|
||||
OpenAI 0.1375 and Microsoft 0.125, Google 0.40 for zero-face content / 0.125 when
|
||||
faces are detected (a content-adaptive arm from the clean face-count split
|
||||
in the calibration: both text cards need 0.25, both face fixtures clear at
|
||||
0.12, so the YuNet detector -- already loaded for the face stage -- routes
|
||||
@@ -1806,9 +1805,11 @@ does not ship: first-cleans from 0.03 to 0.10 form a continuum,
|
||||
`flat_ratio` overlaps the hard and easy clusters, and a high-flat
|
||||
easy-arm would misroute the harvest-1 portraits. The worst seed-0
|
||||
first-clean is still 0.10; botanical seeds 1 and 2 stay DETECTED at
|
||||
that rung, which is the margin the 0.17 floor already holds. OpenAI stays on the flat 0.09 floor: extra SynthID-positive carriers
|
||||
clear at or below 0.06, including a 2-face file, and only the 9-face
|
||||
grid needs 0.075 (seed-stable on 0, 1, 2 via openai.com/research/verify).
|
||||
that rung, which is the margin the 0.17 floor already holds. A later OpenAI
|
||||
holdout invalidated the flat 0.09 conclusion: two carriers first cleared under
|
||||
Chroma at 0.10625 and 0.1375, while qwen-zimage cleared both at its existing
|
||||
0.07675 operating point. Explicit Chroma now uses 0.1375 and `auto` routes
|
||||
OpenAI to qwen-zimage.
|
||||
|
||||
A separate content-balanced check on 2026-08-31 tested the exact production
|
||||
OpenAI and Meta floors over 19 prompt-matched image strata per provider. Each
|
||||
|
||||
+1
-1
@@ -729,7 +729,7 @@ engine = InvisibleEngine(pipeline="sdxl-zimage")
|
||||
|
||||
The `qwen-zimage` extra is required for every profile, including `auto`: each
|
||||
concrete engine runs the same DiffSynth Z-Image face stage. `pipeline="auto"`
|
||||
selects chroma-zimage for OpenAI and Microsoft provenance and qwen-zimage
|
||||
selects chroma-zimage for Microsoft provenance and qwen-zimage
|
||||
otherwise, after the vendor is known and before strength resolution.
|
||||
|
||||
The opt-in verified-text stage uses the same `text_manifest` argument as the CLI:
|
||||
|
||||
@@ -152,7 +152,7 @@ Current pipeline values, all CUDA-only:
|
||||
- `qwen-zimage`, the default;
|
||||
- `sdxl-zimage`, the same recipe and the same face stage on an SDXL global pass;
|
||||
- `chroma-zimage`, the same face stage on a Chroma1 global pass;
|
||||
- `auto`, chroma-zimage for OpenAI and Microsoft provenance, otherwise qwen-zimage.
|
||||
- `auto`, chroma-zimage for Microsoft provenance, otherwise qwen-zimage.
|
||||
|
||||
The `controlnet`, `sdxl`, `qwen` and `default` values were removed. A retired name
|
||||
is rejected at parse time rather than remapped onto a surviving profile.
|
||||
|
||||
@@ -85,7 +85,7 @@ run on.
|
||||
Pipeline profiles: `qwen-zimage` (the default), `sdxl-zimage`, `chroma-zimage`,
|
||||
`auto`. All are CUDA-only and all install from the same `qwen-zimage` extra.
|
||||
`auto` picks the engine from the file's own provenance -- chroma-zimage for
|
||||
OpenAI and Microsoft, qwen-zimage for Google, Meta and unknown -- so prefer it
|
||||
Microsoft, qwen-zimage for OpenAI, Google, Meta and unknown -- so prefer it
|
||||
over naming an engine when the user has no reason to care. `sdxl-zimage` is the
|
||||
heavier alternative and `chroma-zimage` is the Apache-2.0 global stage.
|
||||
|
||||
|
||||
@@ -149,9 +149,11 @@ _SDXL_ZIMAGE_STRENGTH_BY_VENDOR: dict[str, float] = {
|
||||
# guidance 5.0, four effective steps, seed 0). Flat per-vendor values derived
|
||||
# by the same worst-clean-plus-one-spread rule as the other profiles:
|
||||
#
|
||||
# - OpenAI: first-clean boundaries 0.06 / 0.06 / 0.075 on three fixtures ->
|
||||
# 0.075 + (0.075 - 0.06) = 0.09. Chroma1's OpenAI floor is BELOW qwen's
|
||||
# 0.07675 while its matched-strength fidelity beats qwen on every metric.
|
||||
# - OpenAI: the initial three fixtures first cleared at 0.06 / 0.06 / 0.075,
|
||||
# but two withheld carriers expanded the first-clean boundary to 0.10625 and
|
||||
# 0.1375. Explicit Chroma therefore uses the measured worst-clean rung 0.1375.
|
||||
# Qwen cleared both withheld carriers at its existing 0.07675 operating point,
|
||||
# so auto routes OpenAI to Qwen rather than paying Chroma's higher fidelity cost.
|
||||
# - Microsoft InvisMark: paint-1 (0.06, 0.08], paint-2 <= 0.04, paint-3
|
||||
# (0.06, 0.08] -> 0.08 + (0.08 - 0.04) = 0.12, rounded up to the measured
|
||||
# rung 0.125, oracle-verified clean on both worst sources. BELOW qwen's 0.15.
|
||||
@@ -165,7 +167,7 @@ _SDXL_ZIMAGE_STRENGTH_BY_VENDOR: dict[str, float] = {
|
||||
# 0.10 + (0.10 - 0.03) = 0.17. ABOVE qwen's 0.1; the floors exist because
|
||||
# Chroma1's boundaries scatter wider than qwen's, not because the engine is
|
||||
# worse per unit strength.
|
||||
CHROMA_ZIMAGE_OPENAI_STRENGTH = 0.09
|
||||
CHROMA_ZIMAGE_OPENAI_STRENGTH = 0.1375
|
||||
CHROMA_ZIMAGE_MICROSOFT_STRENGTH = 0.125
|
||||
CHROMA_ZIMAGE_GOOGLE_STRENGTH = 0.40
|
||||
CHROMA_ZIMAGE_META_STRENGTH = 0.17
|
||||
@@ -199,11 +201,10 @@ _ALIASES = {
|
||||
|
||||
# The deterministic per-cohort selection policy: which profile wins on which vendor, from the
|
||||
# 2026-08-29/30 four-cohort calibration (docs/chroma1-engine-research.md).
|
||||
# chroma-zimage has lower floors AND better matched-strength fidelity on OpenAI
|
||||
# and Microsoft; qwen-zimage wins on Google and Meta. Unknown stays on qwen
|
||||
# (the shipped default, conservative).
|
||||
# chroma-zimage has the lower measured Microsoft floor. qwen-zimage wins on
|
||||
# OpenAI, Google, and Meta. Unknown stays on qwen (the shipped default,
|
||||
# conservative).
|
||||
_ENGINE_BY_VENDOR: dict[str, str] = {
|
||||
"openai": CHROMA_ZIMAGE_PROFILE,
|
||||
"microsoft": CHROMA_ZIMAGE_PROFILE,
|
||||
}
|
||||
|
||||
|
||||
@@ -257,9 +257,9 @@ _PIPELINE_HELP = (
|
||||
"Pipeline profile. qwen-zimage (DEFAULT) = Qwen-Image-2512 + Lightning + Canny, "
|
||||
"followed by SAM-masked Z-Image face repair; sdxl-zimage = the same recipe and the "
|
||||
"same face stage on an SDXL global pass, which needs more denoise; chroma-zimage = "
|
||||
"the same face stage on an Apache-2.0 Chroma1 global pass with lower OpenAI and "
|
||||
"Microsoft floors and higher Google/Meta floors; auto = pick the engine from the "
|
||||
"provenance (chroma-zimage for OpenAI/Microsoft, qwen-zimage for Google/Meta/unknown). "
|
||||
"the same face stage on an Apache-2.0 Chroma1 global pass; auto = pick the engine "
|
||||
"from the provenance (chroma-zimage for Microsoft, qwen-zimage for "
|
||||
"OpenAI/Google/Meta/unknown). "
|
||||
"All are CUDA-ONLY -- install the qwen-zimage extra. There is no CPU or MPS profile for "
|
||||
"invisible-watermark removal."
|
||||
)
|
||||
|
||||
@@ -1011,7 +1011,7 @@ def test_chroma_zimage_strength_uses_measured_flat_floors():
|
||||
|
||||
assert normalize_profile("chroma_zimage") == "chroma-zimage"
|
||||
assert resolve_strength(None, "openai", "chroma-zimage") == pytest.approx(CHROMA_ZIMAGE_OPENAI_STRENGTH)
|
||||
assert pytest.approx(0.09) == CHROMA_ZIMAGE_OPENAI_STRENGTH
|
||||
assert pytest.approx(0.1375) == CHROMA_ZIMAGE_OPENAI_STRENGTH
|
||||
assert resolve_strength(None, "microsoft", "chroma-zimage") == pytest.approx(CHROMA_ZIMAGE_MICROSOFT_STRENGTH)
|
||||
assert pytest.approx(0.125) == CHROMA_ZIMAGE_MICROSOFT_STRENGTH
|
||||
assert resolve_strength(None, "google", "chroma-zimage") == pytest.approx(CHROMA_ZIMAGE_GOOGLE_STRENGTH)
|
||||
@@ -1046,7 +1046,7 @@ def test_chroma_zimage_inherits_the_shared_stages_and_only_swaps_the_global():
|
||||
def test_chroma_requested_steps_compensate_for_the_diffusers_truncation():
|
||||
from remove_ai_watermarks._internal.chroma_zimage_pipeline import CHROMA_STEPS, requested_steps
|
||||
|
||||
for strength in (0.09, 0.125, 0.17, 0.40):
|
||||
for strength in (0.1375, 0.125, 0.17, 0.40):
|
||||
steps = requested_steps(CHROMA_STEPS, strength)
|
||||
assert int(steps * strength) >= CHROMA_STEPS
|
||||
assert int(CHROMA_STEPS * strength) < CHROMA_STEPS
|
||||
@@ -1295,7 +1295,7 @@ def test_chroma_google_face_content_gets_the_lower_adaptive_floor():
|
||||
# No face_count hint: conservative (flat floor).
|
||||
assert resolve_strength(None, "google", "chroma-zimage") == pytest.approx(0.40)
|
||||
# Other vendors ignore face_count: the split is Google-SynthID-specific.
|
||||
assert resolve_strength(None, "openai", "chroma-zimage", face_count=5) == pytest.approx(0.09)
|
||||
assert resolve_strength(None, "openai", "chroma-zimage", face_count=5) == pytest.approx(0.1375)
|
||||
assert resolve_strength(None, "meta", "chroma-zimage", face_count=5) == pytest.approx(0.17)
|
||||
# An explicit strength still wins over the adaptive arm.
|
||||
assert resolve_strength(0.20, "google", "chroma-zimage", face_count=17) == pytest.approx(0.20)
|
||||
@@ -1305,7 +1305,7 @@ def test_chroma_google_face_content_gets_the_lower_adaptive_floor():
|
||||
|
||||
def test_auto_profile_routes_to_the_measured_engine_per_vendor():
|
||||
"""--pipeline auto picks the engine the four-cohort calibration supports:
|
||||
chroma-zimage for OpenAI/Microsoft, qwen-zimage for Google/Meta/unknown."""
|
||||
chroma-zimage for Microsoft, qwen-zimage for OpenAI/Google/Meta/unknown."""
|
||||
from remove_ai_watermarks._internal.watermark_profiles import (
|
||||
CHROMA_ZIMAGE_PROFILE,
|
||||
QWEN_ZIMAGE_PROFILE,
|
||||
@@ -1314,7 +1314,7 @@ def test_auto_profile_routes_to_the_measured_engine_per_vendor():
|
||||
)
|
||||
|
||||
assert normalize_profile("auto") == "auto"
|
||||
assert resolve_auto_profile("openai") == CHROMA_ZIMAGE_PROFILE
|
||||
assert resolve_auto_profile("openai") == QWEN_ZIMAGE_PROFILE
|
||||
assert resolve_auto_profile("microsoft") == CHROMA_ZIMAGE_PROFILE
|
||||
assert resolve_auto_profile("google") == QWEN_ZIMAGE_PROFILE
|
||||
assert resolve_auto_profile("meta") == QWEN_ZIMAGE_PROFILE
|
||||
@@ -1323,8 +1323,7 @@ def test_auto_profile_routes_to_the_measured_engine_per_vendor():
|
||||
|
||||
|
||||
def test_auto_profile_strength_uses_the_resolved_engine_floors(tmp_path, monkeypatch):
|
||||
"""After auto resolves to chroma-zimage for an OpenAI file, the strength
|
||||
must be chroma's 0.09 floor, not qwen's 0.07675."""
|
||||
"""Auto resolves OpenAI to Qwen and uses its oracle-verified 0.07675 floor."""
|
||||
from remove_ai_watermarks._internal.watermark_remover import WatermarkRemover
|
||||
|
||||
_mock_watermark_runtime_deps(monkeypatch)
|
||||
@@ -1340,14 +1339,13 @@ def test_auto_profile_strength_uses_the_resolved_engine_floors(tmp_path, monkeyp
|
||||
|
||||
remover.remove_watermark(source, vendor="openai")
|
||||
|
||||
# Auto resolved to chroma-zimage for the OpenAI vendor.
|
||||
assert remover.model_profile == "chroma-zimage"
|
||||
assert remover.model_profile == "qwen-zimage"
|
||||
_, kwargs = runtime.run.call_args
|
||||
assert kwargs["strength"] == pytest.approx(0.09)
|
||||
assert kwargs["strength"] == pytest.approx(0.07675)
|
||||
|
||||
|
||||
def test_auto_profile_text_manifest_uses_the_measured_chroma_engine_once(tmp_path, monkeypatch):
|
||||
"""A manifest does not override the researched OpenAI-to-Chroma decision."""
|
||||
def test_auto_profile_text_manifest_uses_the_measured_qwen_engine_once(tmp_path, monkeypatch):
|
||||
"""A manifest does not override the researched OpenAI-to-Qwen decision."""
|
||||
from remove_ai_watermarks._internal.text_restoration import VerifiedTextLine, VerifiedTextManifest
|
||||
from remove_ai_watermarks._internal.watermark_remover import WatermarkRemover
|
||||
|
||||
@@ -1363,6 +1361,6 @@ def test_auto_profile_text_manifest_uses_the_measured_chroma_engine_once(tmp_pat
|
||||
|
||||
remover.remove_watermark(source, vendor="openai", text_manifest=manifest)
|
||||
|
||||
assert remover.model_profile == "chroma-zimage"
|
||||
assert remover.model_profile == "qwen-zimage"
|
||||
assert runtime.run.call_count == 1
|
||||
assert runtime.run.call_args.kwargs["text_manifest"] is manifest
|
||||
Reference in new issue
Block a user