remove-ai-watermarks/docs/chroma1-engine-research.md

Chroma1 engine research (2026-08-29/30)

Research archive. This page records experiments and decisions from the dates above. Current package behavior is defined by the supported signals, known limitations, and module internals. Nothing here is shipped.

Cited experiment behind issue #88 ("Please integrate Flux 2"): FLUX.2 has no native strength img2img anywhere (BFL reference code or diffusers; its image inputs are reference conditioning and outputs start from pure noise), so the requestable scrub lane in the FLUX family is Chroma1 -- the FLUX.1 architecture re-trained on open data, Apache-2.0 on weights, with a native ChromaImg2ImgPipeline strength parameter. This page records what was measured before any integration decision.

Setup

All Chroma1 runs: lodestones/Chroma1-HD bf16 on Modal H100, seed 0, the neutral scrub prompt (high quality, sharp, detailed, faithful to the original / blurry, lowres, distorted text, garbled text, artifacts), guidance 5.0, and step compensation: Diffusers truncates the step COUNT (int(steps * strength)), so the requested count is scaled to always spend 4 effective denoising steps, the same semantics as sdxl_zimage_pipeline.requested_steps. Without it, strength 0.05 runs a single step and the floor reads artificially high (the first, uncompensated sweep put the OpenAI boundary at (0.05, 0.10]; compensated it is (0.05, 0.06]).

Two Flux-family traps found while wiring the prototype:

• without explicit width/height, the img2img pipeline silently generates 1024x1024 regardless of input aspect;
• dimensions floor to the /16 latent-patch grid, so the result must be resized back to the input size for pixel-exact comparison.

Scripts: scripts/chroma_scrub_prototype.py (first sweep), scripts/engine_quality_price_probe.py (head-to-head with the production qwen-zimage path), scripts/engine_cohort_calibration.py (the cohort ladders below), scripts/chroma_meta_high_rungs.py (the Meta rungs above 0.08), scripts/chroma_muse_expansion.py (harvest-2 Meta API fixtures), scripts/chroma_openai_expansion.py (extra OpenAI SynthID carriers), scripts/chroma_seed_probes.py (seed-1/2 at measured first-cleans). Generated outputs stay outside the repository (out/, gitignored); the durable verdicts are the tables on this page.

Measured floors

Oracle per cohort: OpenAI via openai.com/verify (SynthID read separately from C2PA); Meta via the anonymous meta.ai/identification oracle. Each fixture's rungs were checked ascending and stopped at the first clean verdict.

cohort	fixture	detected at	clean from
OpenAI	typography card 1122x1402	0.05	0.06
OpenAI	9-face grid 1448x1086	0.06	0.075
OpenAI	full-pipeline quality fixture 1122x1402	0.05	0.06
Meta	gen_lighthouse_watercolor 1600x1600	0.08	0.10
Meta	gen_fox_forest 1600x1600	0.06	0.08
Meta	gen_night_city 1920x1280	0.045	0.06
Meta	gen_studio_mug 1600x1600	0.03	0.045
Meta	gen_text_poster 1600x1600	--	<=0.03

Derived operating points by the shipped rule (worst first-clean boundary plus one observed cross-source spread):

• OpenAI: 0.09 (0.075 + (0.075 - 0.06)). Against qwen-zimage 0.07675 and sdxl-zimage 0.15. The typography boundary (0.05, 0.06] is seed-stable across seeds 0, 1, 2.

• Meta: 0.17 (0.10 + (0.10 - 0.03)), oracle-verified clean on the worst fixture at 0.17. Against qwen-zimage 0.1. Chroma needs ~1.7x qwen's strength on Content Seal.

• Google: 0.40, measured 2026-08-30 across all four fixtures. A metadata-stripped original verified as pixel-DETECTED through the @synthid Verify AI agent (the C2PA-carrying original answers about C2PA instead, which is not a pixel verdict).

  fixture	detected at	clean from
  Gemini_Generated_Image_633uuy	0.20	0.25
  Gemini_Generated_Image_akdbei	0.20	0.25
  Gemini_Generated_Image_y48j3c	0.08	0.12
  Gemini_Generated_Image_3mc4t9	0.08	0.12

  Derived by the shipped rule: worst first-clean 0.25 + spread (0.25 - 0.12 = 0.13) = 0.38, rounded up to the measured rung 0.40, then oracle-verified clean on the worst fixture. Against qwen-zimage 0.27: Chroma needs ~1.5x the strength on Google SynthID. At 0.15-0.20 the tool still reads positive while its wording degrades from "most or all" to "part of", so the boundary band is real signal decay, not noise.

  Oracle mechanics learned while bracketing: the quota is per ACCOUNT and additionally dedupes SIMILAR images ("quota for checking images similar to this one"), roughly three checks per account window; switching accounts inside one browser works. A fresh Playwright context with the same Google session cookies (exported browser-wide via CDP Storage.getCookies, never through the transcript) carries working sign-in and its own check budget. The agent's own instructions leak into the answer on some renders and include an abstention rule for text-dominated images - always read the TOOL outcome, never the prose.

• Microsoft: 0.125, measured 2026-08-30 on the SAME three valid Paint carriers behind the qwen floor (staged under out/cohort-calibration/microsoft/ with pixel-identical metadata-stripped controls; they are real raiw-corpus user uploads and are never committed). Controls re-verified Microsoft AI detected on the day of the run. Against qwen-zimage 0.15: Chroma needs LESS strength on InvisMark.

  source (Paint)	detected at	clean from	qwen first-clean
  paint-1 (47d87e99, 1536x1024)	0.06	0.08	0.055
  paint-2 (a579fb70, 1536x1024)	--	<=0.04	0.04125
  paint-3 (883191b5, worst qwen source)	0.06	0.08	0.095

  Derived by the shipped rule: 0.08 + (0.08 - 0.04) = 0.12, rounded to the measured rung 0.125, which was then oracle-verified clean on both worst sources. Oracle: the public ai.azure.com/nextgen/validate page; its Inconclusive verdict is the site's watermark-negative, weaker than an API-level Watermark: false, the same caveat the qwen cohort carries.

Fidelity at the floors (scripts/fidelity_metrics.py)

qwen-zimage rows ran through the package's own QwenZImagePipeline on the same H100 (production path, both stacks resident, prompt cache warm).

OpenAI typography (chroma 0.06 vs qwen 0.07675):

	CER	img LPIPS	SSIM	PSNR
qwen-zimage	0.241	0.083	0.834	27.7
chroma1	0.138	0.049	0.881	31.2

OpenAI full-pipeline fixture (chroma 0.09 vs qwen 0.07675): chroma wins every metric (CER 0.155 vs 0.259, LPIPS 0.056 vs 0.094, PSNR 29.6 vs 26.1).

OpenAI 9-face grid (chroma 0.075 global-only vs qwen 0.07675 with its Z-Image face stage):

	ID cos	face LPIPS	img LPIPS	SSIM	PSNR
qwen-zimage	0.857	0.051	0.065	0.865	31.1
chroma1 (no face stage)	0.788	0.067	0.060	0.890	32.4

The face stage regenerates from the ORIGINAL crops and is engine-independent, so an integrated chroma-zimage would composite the same faces over the chroma global; whether that closes the identity gap is an integration-time measurement, not a given.

Meta at the respective floors (chroma 0.17 vs qwen 0.1): qwen wins perceptual fidelity on textured content (LPIPS: fox 0.163 vs 0.211, lighthouse 0.173 vs 0.248, night city 0.187 vs 0.277), chroma wins the text poster decisively (LPIPS 0.008 vs 0.012, CER 0.000 vs 0.021) and the studio mug is a split. The extra 0.07 of strength costs real fidelity on photoreal content.

Google at the respective floors (chroma 0.40 vs qwen 0.27, qwen row includes its Z-Image face stage):

fixture	qwen LPIPS/SSIM/ID cos	chroma LPIPS/SSIM/ID cos
633uuy	0.405 / 0.577 / -	0.499 / 0.576 / - (CER 1.000 vs 0.333)
akdbei	0.440 / 0.552 / -	0.540 / 0.557 / - (CER 0.796 vs 0.367)
y48j3c (18 faces)	0.436 / 0.608 / 0.801	0.560 / 0.548 / 0.279
3mc4t9 (5 faces)	0.422 / 0.529 / 0.921	0.562 / 0.461 / 0.143

qwen wins every axis that matters on every fixture; at 0.40 the Chroma global regeneration itself collapses face identity (0.14-0.28 cosine) and destroys dense text (CER up to 1.0). The extra 0.13 of strength is simply too much regeneration. This is the Meta pattern amplified.

Microsoft at the respective floors (chroma 0.125 vs qwen 0.15): chroma wins EVERY metric on EVERY source while scrubbing at lower strength --

source	CER q/ch	img LPIPS q/ch	SSIM q/ch	PSNR q/ch
paint-1	0.741 / 0.241	0.164 / 0.106	0.742 / 0.847	24.8 / 29.3
paint-2	0.826 / 0.391	0.253 / 0.108	0.536 / 0.633	22.5 / 25.5
paint-3	0.000 / 0.000	0.038 / 0.020	0.762 / 0.953	36.3 / 42.2

The Microsoft cohort therefore behaves like the OpenAI cohort, not like Meta: for InvisMark carriers Chroma1 is the better engine at a lower operating point.

Price (warm H100 container)

	qwen-zimage	chroma1
seconds per image	2.44	3.62
VRAM peak	31.9 GiB	29.2 GiB
USD per 1000 images ($3.95/h)	$2.68	$3.97
weights in cache	~70 GB	~28 GB

The chroma number includes a T5 encode per call; the shipped qwen path caches prompt embeddings, and the same treatment would put chroma near 1.5-2 s/image. Both stacks fit an 80 GB card; chroma's smaller footprint likely fits smaller cards untested.

Matched-strength comparison: the floors hide the engine (2026-08-30 addendum)

The floor-based tables above compare chroma at its DERIVED floor (worst case

• spread) against qwen at its shipped floor - different perturbation levels. At the strength where each fixture ACTUALLY turns clean (chroma first-clean rungs, all oracle-verified above), the picture reverses:

Google (chroma@first-clean vs qwen@0.27):

fixture	chroma rung	LPIPS q/ch	SSIM q/ch	PSNR q/ch	CER q/ch	ID q/ch
633uuy	0.25	0.405/0.375	0.577/0.673	18.8/22.3	0.333/0.667	-
akdbei	0.25	0.440/0.409	0.552/0.650	18.6/22.5	0.367/0.163	-
y48j3c	0.12	0.436/0.329	0.608/0.700	21.0/25.8	1.0/1.0	0.801/0.611
3mc4t9	0.12	0.422/0.318	0.529/0.644	17.7/21.3	0.443/0.229	0.921/0.410

Meta (chroma@first-clean vs qwen@0.1):

fixture	chroma rung	LPIPS q/ch	SSIM q/ch	PSNR q/ch	CER q/ch
lighthouse	0.10 (equal)	0.173/0.097	0.783/0.886	25.6/30.8	-
fox	0.08	0.163/0.083	0.778/0.907	25.6/31.4	-
night_city	0.06	0.187/0.092	0.748/0.893	23.9/29.6	0.850/0.250
studio_mug	0.045	0.024/0.009	0.984/0.992	39.0/37.8	-
text_poster	0.03	0.012/0.002	0.984/0.994	34.0/41.2	0.021/0.007

At effective-clean strength Chroma1 wins whole-image fidelity on essentially every fixture (often by 4-6 dB) and text on five of six; qwen's remaining edge is face identity, which its Z-Image face stage buys (0.80-0.92 vs 0.41-0.61 cosine) and a chroma-zimage profile would inherit on top of its own global pass.

Why the floors still matter: a shipped strength must clear the worst unseen content, and Chroma1's per-fixture boundaries scatter much wider (Google 0.12-0.25, Meta 0.03-0.10) than the spread the flat floors were built to absorb - so its derived floors (0.40 / 0.17) pay a large margin penalty and at those floors the extra strength genuinely destroys quality. The engines differ less in "quality per strength" than in how predictable their boundary is. The integration lever this exposes: a content-adaptive strength policy for chroma-zimage (the project already ships resolution-adaptive denoise for qwen) could capture chroma's low per-content boundaries; the two hard Google fixtures are both dense-text cards and the easy ones are photoreal, so a content-class signal may predict the rung.

Pre-ship validation (2026-08-30)

With the chroma-zimage profile implemented (_internal/chroma_zimage_pipeline.py on TwoStageZImagePipeline, floors in watermark_profiles), two ship-gate checks were run:

1. Seed stability at the worst Meta boundary. lighthouse @ 0.10 with seeds 1 and 2: both CLEAN on the meta.ai oracle. The Meta boundary is not a seed-0 fluke. The Google seed probes (633uuy @ 0.25) delivered a critical finding: seed 0 was CLEAN but seed 1 was DETECTED -- the 633uuy boundary is seed-dependent, which validates the flat 0.40 floor's 0.15 margin exactly the way the qwen SDXL precedent did (the same image at the same strength passing or failing run-to-run near the threshold). Seed 2 of seedprobe_google_633uuy_s0.25_seed2.png was checked twice on 2026-08-31 after a human drag-drop. Work Gemini (u/2, Pro, https://gemini.google.com/u/2/app/ebc7c19d22f85bb9) with the ordinary question "Is there a SynthID watermark in this image?" auto-titled "SynthID Watermark Detection Results" and returned "No reliable signals were detected indicating how the content was created, and you should look for other supporting evidence from trusted sources." Plus @synthid (u/0, https://gemini.google.com/u/0/app/658b1470307811b7) with the built-in verification question auto-titled "SynthID Image Verification Results" and returned "No digital watermark from Google AI was detected in this image, indicating that it was not created or edited using Google's AI models." Both are CLEAN. An earlier Plus @synthid attempt on the same file hung on "Connecting to Verify AI" with auto-title "SynthID Image Verification Failed"; Flash prose abstained because "the tool response does not confirm" Google AI, which is not a tool verdict. Workspace Gemini's More-uploads menu does not list Photos, but a human drag-drop of a local PNG does attach; one Workspace account in the same Chrome has no Gemini app. Seed 0 CLEAN / seed 1 DETECTED / seed 2 CLEAN at 0.25 leaves the 633uuy boundary seed-dependent on the Plus @synthid path; the 0.40 floor is already established by the seed-1 flip.
2. Full-path face-stage interaction. The complete profile run (Chroma1 global at the shipped Google floor 0.40, then the inherited YuNet/SAM/Z-Image face stage) on the 5-face fixture 3mc4t9: CLEAN on the @synthid oracle. The composited face regions do not re-introduce a detectable signal on top of the floor.

Generation script: scripts/chroma_preship_validation.py; outputs under out/preship-chroma/ (gitignored).

Verdict

Engine quality is cohort-dependent, and after measuring all four cohorts the split is even and sharp:

• On OpenAI-provenance content Chroma1 is better at every measured fidelity axis at its floor (0.09 vs 0.07675), for comparable money.
• On Microsoft InvisMark content Chroma1 is better at every measured fidelity axis at a LOWER floor than qwen (0.125 vs 0.15).
• On Meta Content Seal Chroma1 needs 1.7x the strength (0.17 vs 0.1) and loses LPIPS on textured content at its floor.
• On Google SynthID Chroma1 needs ~1.5x the strength (0.40 vs 0.27) and loses every fidelity axis on every fixture, with face identity collapsing at 0.40 (0.14-0.28 cosine vs qwen's 0.80-0.92).

A single-engine chroma-zimage default at a flat per-vendor floor is NOT supported: Google and Meta floors land too high and the quality at those floors is worse than qwen's. But the matched-strength addendum changes what IS supported: at the strength each image actually needs, Chroma1 regenerates better than qwen almost everywhere except face identity (which the shared face stage would supply). The two honest integration options:

1. Per-cohort engine router at flat floors: chroma for OpenAI/Microsoft, qwen for Google/Meta - conservative, ships on the measurements as they stand.
2. chroma-zimage with a content-adaptive strength policy that predicts the per-image rung - potentially better everywhere, but the predictor is new research (the measured signal so far: dense-text content needs the high rungs, photoreal the low ones) and needs its own oracle validation before any ship.

Both need the seed-sensitivity and face-stage composition checks called out above before shipping. Both checks have now been run (Pre-ship validation above): the Meta boundary is seed-stable across seeds 0-2 and the full profile path stays clean through the face stage on the shipped Google floor. The profile ships with option 1's floors plus a content-adaptive Google arm (face content resolves to 0.125 instead of 0.40, keyed on YuNet detection). Option 2's full adaptive policy for the remaining cohorts remains the documented follow-up. The Meta arm was measured on 2026-08-30/31 and does not have a shippable split; see the expansion section below.

Meta content-adaptive expansion (2026-08-30/31)

Goal: test whether flat_ratio (fraction of 16x16 blocks with luma std < 8) predicts Chroma1's first-clean Content Seal boundary well enough to ship a Meta arm analogous to Google's face-count split.

Harvest 1, from a 495-file spaces dump, was not a diverse expansion. Seven "unique" files were staged under out/cohort-calibration/meta/muse-* (gitignored). After control checks on meta.ai/identification:

fixture	control	first-clean	note
muse-1	CLEAN	--	seal did not survive the metadata-stripped control
muse-2	DETECTED	(0.08, 0.10]	photoreal portrait; 0.06 and 0.08 DETECTED, 0.10 CLEAN
muse-3	DETECTED	(0.015, 0.03]	photoreal portrait, different subject
muse-4	DETECTED	(0.08, 0.10]	same subject as muse-2/6/7; 0.06 and 0.08 DETECTED, 0.10 CLEAN
muse-5	--	--	byte-identical to committed gen_lighthouse_watercolor.webp
muse-6	DETECTED	(0.06, 0.08]	same subject as muse-4; 0.06 DETECTED, 0.08 CLEAN
muse-7	DETECTED	(0.045, 0.06]	same subject as muse-2/4/6; 0.045 DETECTED, 0.06 CLEAN

Five valid new points, four of them near-duplicate portraits of one subject, with first-clean already spanning 0.06-0.10 inside that subject. Google's shipped split had zero overlap between classes and identical boundaries inside a class. This harvest cannot support a predictor.

flat_ratio on the original five plus the new portraits is not monotonic with first-clean either (studio_mug 0.973 / 0.045 easy; lighthouse 0.602 / 0.10 hard; muse-4 0.789 / 0.10 hard). OpenAI stays unshippable for the same reason as before: one text fixture vs one face fixture, and the face fixture was the harder one, the opposite of Google's split.

Harvest 2 takes six files from the 2026-08-26 Muse Image API corpus (61 independent muse-image-1.0 generations, no hash overlap with the committed five), chosen to span class and flat_ratio:

fixture	class	flat_ratio	edge_density
architecture_mosque	dense architecture	0.037	0.367
product_sneaker	product / high-flat	0.874	0.018
text_poster_bakery	text	0.368	0.191
portrait_weathered_fisherman	photoreal portrait	0.470	0.108
illustration_watercolor_botanical	illustration	0.442	0.089
scene_tokyo_alley	busy scene	0.258	0.175

Script: scripts/chroma_muse_expansion.py. Outputs under out/cohort-calibration/meta-api/ (gitignored). Chroma1 ladders (seven rungs, seed 0, four effective steps) ran for all six. Oracle 2026-08-31, anonymous playwright-isolated against meta.ai/identification. All six controls DETECTED (valid carriers).

fixture	flat_ratio	detected at	clean from
architecture_mosque	0.037	0.08	0.10
illustration_watercolor_botanical	0.442	0.08	0.10
scene_tokyo_alley	0.258	0.06	0.08
text_poster_bakery	0.368	0.045	0.06
portrait_weathered_fisherman	0.470	0.045	0.06
product_sneaker	0.874	0.03	0.045

The worst first-clean is still 0.10, the same as the committed lighthouse, so the shipped Meta floor 0.17 does not move.

No Meta adaptive arm. Google's shipped split had zero overlap between classes and identical first-cleans inside a class. Combined with the original five, Meta first-cleans are a continuum from 0.03 to 0.10, and flat_ratio does not separate them:

• the 0.10 cluster is mosque 0.037, botanical 0.442, and lighthouse 0.602;
• the 0.045 cluster is sneaker 0.874 and studio_mug 0.973;
• harvest-1 photoreal portraits were high-flat (0.75-0.85) AND hard (0.08-0.10), so a high-flat easy-arm would misroute them.

A content-class rule fails the same way: two text posters first-clean at 0.03 and 0.06, and two watercolor-like images (lighthouse, botanical) share 0.10 with dense architecture.

The leftover rungs were finished in a 2026-08-31 evening UTC anonymous isolated session (fresh-navigation, wait for "Upload another file"; all ten POST /api/ai-detector calls returned 200):

check	verdict
muse-2 @ 0.08	DETECTED
muse-3 @ 0.015 / 0.03 / 0.045	DETECTED / CLEAN / CLEAN
muse-7 @ 0.045 / 0.06	DETECTED / CLEAN
mosque @ 0.10 seeds 1, 2	CLEAN / CLEAN
botanical @ 0.10 seeds 1, 2	DETECTED / DETECTED

botanical's seed-0 first-clean of 0.10 is not seed-stable; lighthouse @ 0.10 seeds 1 and 2 were CLEAN. That is the same class of run-to-run flip the 0.17 floor's margin exists for. No Google-style binary split: harvest-1 first-cleans now span 0.03 (muse-3) through 0.10 (muse-2/4), and 0.06-0.10 inside one subject (muse-7 / muse-6 / muse-2).

OpenAI content-adaptive expansion (2026-08-31)

Goal: test whether face_count or flat_ratio predicts Chroma1's OpenAI SynthID first-clean the way YuNet does for Google.

Committed three (YuNet + flat_ratio on the files, first-cleans from the 2026-08-29/30 web oracle):

fixture	faces	flat_ratio	first-clean
typography 02_03_55	0	0.642	0.06
quality 02_02_23	0	0.695	0.06
9-face grid 05_30	9	0.473	0.075

Face content is the harder of the two classes, the opposite of Google. Two zero-face cards share a boundary, which is a hint, not a split.

Images API is not a carrier source. gpt-image-1 and gpt-image-1.5 generations (medium, 1024) carry C2PA (detected) but SynthID not_detected on POST /v1/content_provenance_checks. Contrastive-pair "openai" recreations are also SynthID-negative. New ChatGPT-UI downloads are not in the tree beyond the three committed files.

Spaces harvest (one day of _full_scan, 2026-07-24): 41 files with OpenAI C2PA, of which 7 checked by the provenance API were SynthID DETECTED (one NOT_DETECTED). Four diverse DETECTED files were staged under out/cohort-calibration/openai-expand/ (gitignored; user uploads, not committed) and given a Chroma1 ladder (0.04-0.12, seed 0) by scripts/chroma_openai_expansion.py:

id	faces	flat_ratio
dense_ed68	0	0.118
midflat_038c	0	0.323
textlike_3b17	0	0.649
faces2_4dab	2	0.279

The provenance API then 429'd. The same checks were finished on https://openai.com/research/verify/ (the page reports SynthID and C2PA separately under View details). Chroma PNG outputs have no C2PA, so a "No OpenAI signals detected" heading is a pixel-SynthID negative.

id	faces	@0.04	@0.05	@0.06
dense_ed68	0	CLEAN	CLEAN	CLEAN
midflat_038c	0	--	--	CLEAN
textlike_3b17	0	--	--	CLEAN
faces2_4dab	2	--	--	CLEAN

9-face grid @ 0.075, seeds 1 and 2: both CLEAN (SynthID not detected). The committed seed-0 first-clean is not a fluke.

No OpenAI adaptive arm. Zero-face extras clear at or below 0.06, the 2-face extra also clears at 0.06, and only the 9-face grid needs 0.075. That is not a YuNet split: face_count > 0 would over-strengthen the 2-face file, and a high face-count cutoff would be one fixture. The spread (0.075 - 0.06) is already absorbed by the shipped 0.09 floor. Images API generations remain non-carriers (C2PA only).

Content-balanced engine selection check (2026-08-31)

The earlier tables were intentionally carrier-focused and therefore small. A separate discovery pass tested whether ordinary image content contradicts the cohort choice at the shipped floors. The tracked matrix contains 19 prompts, matched across OpenAI gpt-image-1-mini and Meta muse-image-1.0, spanning photoreal scenes, faces, action, text-like flat art, products, architecture, watercolor, and surreal illustration. These files were re-encoded by collection and are not signal carriers, so this pass measures fidelity only; removal remains established by the provider-oracle calibration above.

Qwen and Chroma ran in separate sequential H100 invocations, once per source, with the production provider floor and fixed seed. The comparison isolated the global stage because both profiles inherit the same source-based Z-Image face repair. Per-image differences, rather than differences between aggregate means, were evaluated with exact two-sided sign tests:

provider	metric	Chroma wins	Qwen wins	two-sided p
OpenAI	SSIM	19	0	0.0000038
OpenAI	PSNR	19	0	0.0000038
OpenAI	MAE	18	1	0.000076
OpenAI	edge F1	15	4	0.019
OpenAI	LPIPS	12	7	0.359
Meta	LPIPS	1	18	0.000076
Meta	edge F1	1	18	0.000076
Meta	SSIM	5	14	0.064

Meta's remaining pixel metrics disagree: Chroma wins PSNR on 16/19 and MAE on 14/19, while losing both perceptual distance and edge preservation on 18/19 at its higher removal floor. This is not evidence for a genre classifier. The direction instead follows the already measured provenance cohort: Chroma for OpenAI, Qwen for Meta. On OpenAI, the seven LPIPS exceptions include unrelated classes (landscape, sport, low-light interior, watercolor, monochrome portrait, surreal art, and brutalism), while Chroma still wins SSIM and PSNR on every one; there is no coherent input feature that justifies overriding the cohort choice.

Inputs and integrity manifests are under data/evaluations/engine-selection/. Raw generated outputs remain gitignored under out/engine-selection-study/; scripts/engine_selection_study.py reproduces the run and scripts/analyze_engine_selection_study.py reproduces the paired statistics. One generation per prompt is a discovery set, not a license to fit thresholds. Any future content override must first predict a rule here and then survive unused generation indices without changing it.

Oracle session notes

• openai.com/verify throttles anonymous sessions through Cloudflare Turnstile after ~15 rapid checks; a fresh browser context clears it. The programmatic POST /v1/content_provenance_checks API is the better oracle (separate SynthID/C2PA outcomes). The OPENAI_API_KEY in the main-checkout .env returned 401 org-wide, including /v1/models, through the 2026-08-30 calibration; it was replaced on 2026-08-31 and both /v1/models and a provenance check on a committed ChatGPT original then succeeded (DETECTED).
• meta.ai/identification enforces a sliding per-IP window: a burst of ~15 checks exhausted it twice; it reopens minutes later, and the page sometimes overstates this as a daily limit. Pacing of roughly 2-3 checks per window worked. On 2026-08-31 an anonymous isolated session got a real 429 from POST /api/ai-detector with page text "You’ve reached the daily limit for identifications. Try again tomorrow" after about six checks in the UTC morning and again after about fifteen in the UTC afternoon, so the daily-limit wording is not always overstated; the afternoon window was large enough to finish harvest 2. An evening isolated session the same day got ten 200s and finished the leftover harvest-1 rungs plus the mosque/botanical seed probes.
• The Gemini app oracle requires the user's logged-in session. A human drag-drop of a local PNG works on Work Gemini (u/2, 2026-08-31) and on Plus (u/0 the same day); Playwright cannot drive the OS picker, and the first input[type=file].accept is documents/code only. Plus @synthid hung once after attach, then returned a tool CLEAN on a second human-drag chat. Do not drive the live Chrome while it is in active use (wrong-tab risk).