Route OpenAI auto removal through Qwen

This commit is contained in:
Victor Kuznetsov committed 2026-09-07 14:01:17 -07:00
1 parent e645fac166
commit 723f2949ec
10 files changed
+67 -46

No files matched your search

+26 -6
View File
@@ -59,10 +59,10 @@ rungs were checked ascending and stopped at the first clean verdict.
| Meta | gen_studio_mug 1600x1600 | 0.03 | 0.045 |
| Meta | gen_text_poster 1600x1600 | -- | <=0.03 |
Derived operating points by the shipped rule (worst first-clean boundary plus
one observed cross-source spread):
Derived operating points from this initial calibration (the OpenAI value was
superseded by the 2026-09-07 holdout below):
- **OpenAI: 0.09** (0.075 + (0.075 - 0.06)). Against qwen-zimage 0.07675 and
- **OpenAI: 0.09, superseded.** (0.075 + (0.075 - 0.06)). Against qwen-zimage 0.07675 and
sdxl-zimage 0.15. The typography boundary (0.05, 0.06] is seed-stable across
seeds 0, 1, 2.
- **Meta: 0.17** (0.10 + (0.10 - 0.03)), oracle-verified clean on the worst
@@ -284,8 +284,8 @@ Generation script: `scripts/chroma_preship_validation.py`; outputs under
Engine quality is cohort-dependent, and after measuring all four cohorts the
split is even and sharp:
- On OpenAI-provenance content Chroma1 is better at every measured fidelity
axis at its floor (0.09 vs 0.07675), for comparable money.
- On the initial OpenAI fixtures Chroma1 was better at the then-current floor,
but the later holdout moved that floor to 0.1375 and changed the router decision.
- On Microsoft InvisMark content Chroma1 is better at every measured fidelity
axis at a LOWER floor than qwen (0.125 vs 0.15).
- On Meta Content Seal Chroma1 needs 1.7x the strength (0.17 vs 0.1) and loses
@@ -473,9 +473,29 @@ The committed seed-0 first-clean is not a fluke.
2-face extra also clears at 0.06, and only the 9-face grid needs 0.075.
That is not a YuNet split: `face_count > 0` would over-strengthen the
2-face file, and a high face-count cutoff would be one fixture. The
spread (0.075 - 0.06) is already absorbed by the shipped 0.09 floor.
spread (0.075 - 0.06) was absorbed by the 0.09 floor for this calibration,
but the later holdout below invalidated that operating point.
Images API generations remain non-carriers (C2PA only).
## OpenAI hard-carrier holdout (2026-09-07)
Two additional withheld OpenAI SynthID carriers expanded the Chroma boundary.
Each original was a positive pixel control after metadata-only stripping, so the
verdict did not depend on C2PA provenance. Their Chroma first-clean strengths were
0.10625 and 0.1375. Both Qwen outputs were clean at the existing 0.07675 operating
point.
The paired fidelity read was mixed rather than a reason to pay Chroma's higher
denoise. On the one detected face, Qwen had higher identity cosine (0.835 vs
0.821), lower face LPIPS (0.108 vs 0.118), and lower whole-image LPIPS (0.095 vs
0.125); Chroma retained more local texture and had slightly higher SSIM. On the
second carrier Qwen again had slightly lower LPIPS (0.104 vs 0.109), while Chroma
had higher SSIM and PSNR.
This invalidates both the 0.09 explicit Chroma floor and the OpenAI auto route.
Explicit `chroma-zimage` now uses the measured worst-clean rung 0.1375. `auto`
uses `qwen-zimage` for OpenAI at 0.07675; Chroma remains the Microsoft route.
## Content-balanced engine selection check (2026-08-31)
The earlier tables were intentionally carrier-focused and therefore small. A
+2 -2
View File
@@ -482,8 +482,8 @@ The same option exists on `all` and `batch`, and as
| --- | --- |
| `qwen-zimage` | Default. Qwen-Image-2512 global pass plus a SAM-masked Z-Image face stage |
| `sdxl-zimage` | The same recipe and face stage on an SDXL global pass, at a higher denoise |
| `chroma-zimage` | The same face stage on an Apache-2.0 Chroma1 global pass, with lower OpenAI and Microsoft floors and higher Google and Meta floors |
| `auto` | Pick the engine from provenance: chroma-zimage for OpenAI and Microsoft, qwen-zimage for Google, Meta, and unknown |
| `chroma-zimage` | The same face stage on an Apache-2.0 Chroma1 global pass; explicit OpenAI use needs a higher floor than qwen-zimage |
| `auto` | Pick the engine from provenance: chroma-zimage for Microsoft, qwen-zimage for OpenAI, Google, Meta, and unknown |
**All four are CUDA-only.** There is no CPU or MPS profile for invisible-watermark
removal. The former `controlnet`, `sdxl`, `qwen` and `default` profiles were removed
+5 -4
View File
@@ -105,7 +105,7 @@ frozen text regions would keep the watermark
`qwen-zimage` is the default profile. `sdxl-zimage` and `chroma-zimage` keep
the same face stage and swap the global model. `auto` picks chroma-zimage for
OpenAI and Microsoft provenance and qwen-zimage otherwise. All are CUDA only.
Microsoft provenance and qwen-zimage otherwise. All are CUDA only.
Qwen and SDXL condition the global stage on a canny edge map, which preserves
structure but not identity or exact texture. Chroma1 is a plain strength pass
without Canny.
@@ -251,8 +251,9 @@ established for that stage.
2026-08-29/30 across the four vendor oracles (full record:
[`chroma1-engine-research.md`](chroma1-engine-research.md)):
- OpenAI: `0.09` (below qwen's `0.07675` is false economy neither way -- the
matched-strength fidelity favors Chroma1 on this cohort);
- OpenAI: `0.1375`. Two withheld carriers expanded the Chroma first-clean
boundary beyond the original `0.09` policy while qwen-zimage cleared both at
its existing `0.07675` operating point;
- Microsoft InvisMark: `0.125` (below qwen's `0.15`);
- Google: `0.40` for zero-face content (above qwen's `0.27`; at this floor the
regeneration destroys dense text and collapses face identity -- the tradeoff
@@ -275,7 +276,7 @@ are certified at a fixed seed. The live resolver is
| `qwen-zimage` | CUDA only, large model stack, and limited broad certification across seeds and content. |
| `sdxl-zimage` | CUDA only. Its strength ladder is flat per vendor, not a resolution curve, because flat values are what was measured. |
| `chroma-zimage` | CUDA only. Higher Google and Meta floors than qwen; Google face content uses 0.125 instead of 0.40. |
| `auto` | CUDA only. Routes OpenAI and Microsoft to chroma-zimage and everything else to qwen-zimage. |
| `auto` | CUDA only. Routes Microsoft to chroma-zimage and everything else to qwen-zimage. |
Only manually verified `vae-glyphs` is an optional production stage, and it is
experimental rather than a default.
+8 -7
View File
@@ -1520,8 +1520,8 @@ The current profiles are `qwen-zimage` (the default), `sdxl-zimage`,
`qwen` and `default` were removed rather than kept as a CPU path, and are
rejected rather than aliased onward. `auto` is a deterministic per-cohort
selection policy:
chroma-zimage for OpenAI and Microsoft, qwen-zimage for Google, Meta, and
unknown. It does not run a learned router or classify the image's genre.
chroma-zimage for Microsoft, qwen-zimage for OpenAI, Google, Meta, and unknown.
It does not run a learned router or classify the image's genre.
`qwen-zimage` normally resolves global denoise from image area for unknown content.
Measured provider cohorts bypass that curve with flat operating points. The values,
@@ -1789,8 +1789,7 @@ Diffusers truncation trap sdxl-zimage documents). There is no Canny
conditioning: the floors were measured on a plain strength pass.
The flat vendor floors (`CHROMA_ZIMAGE_*_STRENGTH` in watermark_profiles):
OpenAI 0.09 and Microsoft 0.125 (both BELOW qwen's, with better
matched-strength fidelity), Google 0.40 for zero-face content / 0.125 when
OpenAI 0.1375 and Microsoft 0.125, Google 0.40 for zero-face content / 0.125 when
faces are detected (a content-adaptive arm from the clean face-count split
in the calibration: both text cards need 0.25, both face fixtures clear at
0.12, so the YuNet detector -- already loaded for the face stage -- routes
@@ -1806,9 +1805,11 @@ does not ship: first-cleans from 0.03 to 0.10 form a continuum,
`flat_ratio` overlaps the hard and easy clusters, and a high-flat
easy-arm would misroute the harvest-1 portraits. The worst seed-0
first-clean is still 0.10; botanical seeds 1 and 2 stay DETECTED at
that rung, which is the margin the 0.17 floor already holds. OpenAI stays on the flat 0.09 floor: extra SynthID-positive carriers
clear at or below 0.06, including a 2-face file, and only the 9-face
grid needs 0.075 (seed-stable on 0, 1, 2 via openai.com/research/verify).
that rung, which is the margin the 0.17 floor already holds. A later OpenAI
holdout invalidated the flat 0.09 conclusion: two carriers first cleared under
Chroma at 0.10625 and 0.1375, while qwen-zimage cleared both at its existing
0.07675 operating point. Explicit Chroma now uses 0.1375 and `auto` routes
OpenAI to qwen-zimage.
A separate content-balanced check on 2026-08-31 tested the exact production
OpenAI and Meta floors over 19 prompt-matched image strata per provider. Each
+1 -1
View File
@@ -729,7 +729,7 @@ engine = InvisibleEngine(pipeline="sdxl-zimage")
The `qwen-zimage` extra is required for every profile, including `auto`: each
concrete engine runs the same DiffSynth Z-Image face stage. `pipeline="auto"`
selects chroma-zimage for OpenAI and Microsoft provenance and qwen-zimage
selects chroma-zimage for Microsoft provenance and qwen-zimage
otherwise, after the vendor is known and before strength resolution.
The opt-in verified-text stage uses the same `text_manifest` argument as the CLI:
+1 -1
View File
@@ -152,7 +152,7 @@ Current pipeline values, all CUDA-only:
- `qwen-zimage`, the default;
- `sdxl-zimage`, the same recipe and the same face stage on an SDXL global pass;
- `chroma-zimage`, the same face stage on a Chroma1 global pass;
- `auto`, chroma-zimage for OpenAI and Microsoft provenance, otherwise qwen-zimage.
- `auto`, chroma-zimage for Microsoft provenance, otherwise qwen-zimage.
The `controlnet`, `sdxl`, `qwen` and `default` values were removed. A retired name
is rejected at parse time rather than remapped onto a surviving profile.
@@ -85,7 +85,7 @@ run on.
Pipeline profiles: `qwen-zimage` (the default), `sdxl-zimage`, `chroma-zimage`,
`auto`. All are CUDA-only and all install from the same `qwen-zimage` extra.
`auto` picks the engine from the file's own provenance -- chroma-zimage for
OpenAI and Microsoft, qwen-zimage for Google, Meta and unknown -- so prefer it
Microsoft, qwen-zimage for OpenAI, Google, Meta and unknown -- so prefer it
over naming an engine when the user has no reason to care. `sdxl-zimage` is the
heavier alternative and `chroma-zimage` is the Apache-2.0 global stage.
@@ -149,9 +149,11 @@ _SDXL_ZIMAGE_STRENGTH_BY_VENDOR: dict[str, float] = {
# guidance 5.0, four effective steps, seed 0). Flat per-vendor values derived
# by the same worst-clean-plus-one-spread rule as the other profiles:
#
# - OpenAI: first-clean boundaries 0.06 / 0.06 / 0.075 on three fixtures ->
# 0.075 + (0.075 - 0.06) = 0.09. Chroma1's OpenAI floor is BELOW qwen's
# 0.07675 while its matched-strength fidelity beats qwen on every metric.
# - OpenAI: the initial three fixtures first cleared at 0.06 / 0.06 / 0.075,
# but two withheld carriers expanded the first-clean boundary to 0.10625 and
# 0.1375. Explicit Chroma therefore uses the measured worst-clean rung 0.1375.
# Qwen cleared both withheld carriers at its existing 0.07675 operating point,
# so auto routes OpenAI to Qwen rather than paying Chroma's higher fidelity cost.
# - Microsoft InvisMark: paint-1 (0.06, 0.08], paint-2 <= 0.04, paint-3
# (0.06, 0.08] -> 0.08 + (0.08 - 0.04) = 0.12, rounded up to the measured
# rung 0.125, oracle-verified clean on both worst sources. BELOW qwen's 0.15.
@@ -165,7 +167,7 @@ _SDXL_ZIMAGE_STRENGTH_BY_VENDOR: dict[str, float] = {
# 0.10 + (0.10 - 0.03) = 0.17. ABOVE qwen's 0.1; the floors exist because
# Chroma1's boundaries scatter wider than qwen's, not because the engine is
# worse per unit strength.
CHROMA_ZIMAGE_OPENAI_STRENGTH = 0.09
CHROMA_ZIMAGE_OPENAI_STRENGTH = 0.1375
CHROMA_ZIMAGE_MICROSOFT_STRENGTH = 0.125
CHROMA_ZIMAGE_GOOGLE_STRENGTH = 0.40
CHROMA_ZIMAGE_META_STRENGTH = 0.17
@@ -199,11 +201,10 @@ _ALIASES = {
# The deterministic per-cohort selection policy: which profile wins on which vendor, from the
# 2026-08-29/30 four-cohort calibration (docs/chroma1-engine-research.md).
# chroma-zimage has lower floors AND better matched-strength fidelity on OpenAI
# and Microsoft; qwen-zimage wins on Google and Meta. Unknown stays on qwen
# (the shipped default, conservative).
# chroma-zimage has the lower measured Microsoft floor. qwen-zimage wins on
# OpenAI, Google, and Meta. Unknown stays on qwen (the shipped default,
# conservative).
_ENGINE_BY_VENDOR: dict[str, str] = {
"openai": CHROMA_ZIMAGE_PROFILE,
"microsoft": CHROMA_ZIMAGE_PROFILE,
}
+3 -3
View File
@@ -257,9 +257,9 @@ _PIPELINE_HELP = (
"Pipeline profile. qwen-zimage (DEFAULT) = Qwen-Image-2512 + Lightning + Canny, "
"followed by SAM-masked Z-Image face repair; sdxl-zimage = the same recipe and the "
"same face stage on an SDXL global pass, which needs more denoise; chroma-zimage = "
"the same face stage on an Apache-2.0 Chroma1 global pass with lower OpenAI and "
"Microsoft floors and higher Google/Meta floors; auto = pick the engine from the "
"provenance (chroma-zimage for OpenAI/Microsoft, qwen-zimage for Google/Meta/unknown). "
"the same face stage on an Apache-2.0 Chroma1 global pass; auto = pick the engine "
"from the provenance (chroma-zimage for Microsoft, qwen-zimage for "
"OpenAI/Google/Meta/unknown). "
"All are CUDA-ONLY -- install the qwen-zimage extra. There is no CPU or MPS profile for "
"invisible-watermark removal."
)
+11 -13
View File
@@ -1011,7 +1011,7 @@ def test_chroma_zimage_strength_uses_measured_flat_floors():
assert normalize_profile("chroma_zimage") == "chroma-zimage"
assert resolve_strength(None, "openai", "chroma-zimage") == pytest.approx(CHROMA_ZIMAGE_OPENAI_STRENGTH)
assert pytest.approx(0.09) == CHROMA_ZIMAGE_OPENAI_STRENGTH
assert pytest.approx(0.1375) == CHROMA_ZIMAGE_OPENAI_STRENGTH
assert resolve_strength(None, "microsoft", "chroma-zimage") == pytest.approx(CHROMA_ZIMAGE_MICROSOFT_STRENGTH)
assert pytest.approx(0.125) == CHROMA_ZIMAGE_MICROSOFT_STRENGTH
assert resolve_strength(None, "google", "chroma-zimage") == pytest.approx(CHROMA_ZIMAGE_GOOGLE_STRENGTH)
@@ -1046,7 +1046,7 @@ def test_chroma_zimage_inherits_the_shared_stages_and_only_swaps_the_global():
def test_chroma_requested_steps_compensate_for_the_diffusers_truncation():
from remove_ai_watermarks._internal.chroma_zimage_pipeline import CHROMA_STEPS, requested_steps
for strength in (0.09, 0.125, 0.17, 0.40):
for strength in (0.1375, 0.125, 0.17, 0.40):
steps = requested_steps(CHROMA_STEPS, strength)
assert int(steps * strength) >= CHROMA_STEPS
assert int(CHROMA_STEPS * strength) < CHROMA_STEPS
@@ -1295,7 +1295,7 @@ def test_chroma_google_face_content_gets_the_lower_adaptive_floor():
# No face_count hint: conservative (flat floor).
assert resolve_strength(None, "google", "chroma-zimage") == pytest.approx(0.40)
# Other vendors ignore face_count: the split is Google-SynthID-specific.
assert resolve_strength(None, "openai", "chroma-zimage", face_count=5) == pytest.approx(0.09)
assert resolve_strength(None, "openai", "chroma-zimage", face_count=5) == pytest.approx(0.1375)
assert resolve_strength(None, "meta", "chroma-zimage", face_count=5) == pytest.approx(0.17)
# An explicit strength still wins over the adaptive arm.
assert resolve_strength(0.20, "google", "chroma-zimage", face_count=17) == pytest.approx(0.20)
@@ -1305,7 +1305,7 @@ def test_chroma_google_face_content_gets_the_lower_adaptive_floor():
def test_auto_profile_routes_to_the_measured_engine_per_vendor():
"""--pipeline auto picks the engine the four-cohort calibration supports:
chroma-zimage for OpenAI/Microsoft, qwen-zimage for Google/Meta/unknown."""
chroma-zimage for Microsoft, qwen-zimage for OpenAI/Google/Meta/unknown."""
from remove_ai_watermarks._internal.watermark_profiles import (
CHROMA_ZIMAGE_PROFILE,
QWEN_ZIMAGE_PROFILE,
@@ -1314,7 +1314,7 @@ def test_auto_profile_routes_to_the_measured_engine_per_vendor():
)
assert normalize_profile("auto") == "auto"
assert resolve_auto_profile("openai") == CHROMA_ZIMAGE_PROFILE
assert resolve_auto_profile("openai") == QWEN_ZIMAGE_PROFILE
assert resolve_auto_profile("microsoft") == CHROMA_ZIMAGE_PROFILE
assert resolve_auto_profile("google") == QWEN_ZIMAGE_PROFILE
assert resolve_auto_profile("meta") == QWEN_ZIMAGE_PROFILE
@@ -1323,8 +1323,7 @@ def test_auto_profile_routes_to_the_measured_engine_per_vendor():
def test_auto_profile_strength_uses_the_resolved_engine_floors(tmp_path, monkeypatch):
"""After auto resolves to chroma-zimage for an OpenAI file, the strength
must be chroma's 0.09 floor, not qwen's 0.07675."""
"""Auto resolves OpenAI to Qwen and uses its oracle-verified 0.07675 floor."""
from remove_ai_watermarks._internal.watermark_remover import WatermarkRemover
_mock_watermark_runtime_deps(monkeypatch)
@@ -1340,14 +1339,13 @@ def test_auto_profile_strength_uses_the_resolved_engine_floors(tmp_path, monkeyp
remover.remove_watermark(source, vendor="openai")
# Auto resolved to chroma-zimage for the OpenAI vendor.
assert remover.model_profile == "chroma-zimage"
assert remover.model_profile == "qwen-zimage"
_, kwargs = runtime.run.call_args
assert kwargs["strength"] == pytest.approx(0.09)
assert kwargs["strength"] == pytest.approx(0.07675)
def test_auto_profile_text_manifest_uses_the_measured_chroma_engine_once(tmp_path, monkeypatch):
"""A manifest does not override the researched OpenAI-to-Chroma decision."""
def test_auto_profile_text_manifest_uses_the_measured_qwen_engine_once(tmp_path, monkeypatch):
"""A manifest does not override the researched OpenAI-to-Qwen decision."""
from remove_ai_watermarks._internal.text_restoration import VerifiedTextLine, VerifiedTextManifest
from remove_ai_watermarks._internal.watermark_remover import WatermarkRemover
@@ -1363,6 +1361,6 @@ def test_auto_profile_text_manifest_uses_the_measured_chroma_engine_once(tmp_pat
remover.remove_watermark(source, vendor="openai", text_manifest=manifest)
assert remover.model_profile == "chroma-zimage"
assert remover.model_profile == "qwen-zimage"
assert runtime.run.call_count == 1
assert runtime.run.call_args.kwargs["text_manifest"] is manifest