# Photo classifier training and data

How the 2026-08-31 freeze was fit, what a retrain needs, and where those
files come from. Inference is [photo-classify.md](photo-classify.md). The
campaign log is [ai-generated-image-classifiers.md](ai-generated-image-classifiers.md).

The shipped package does not include the catalog or the embeddings. Fitting
the heads again uses the cache pack. The original pixels still have to be
kept: a missing embedding, a new backbone, or an audit all need the files.

## Retrain pack (caches)

CPU retrain of both heads needs sha256-keyed matrices and the catalog of
roles. That step does not open JPEG or PNG files. It is not permission to
drop the pixels.

| File | Role |
| --- | --- |
| `catalog.json` | 32,690 rows: `sha256`, `roles`, optional `family` / public `source` |
| `embeddings-by-sha256.npz` | CLIP-L-ft 768-d vectors for Model 1 |
| `extra-clip-l.npz` | CLIP-L extras for the 150 Flickr UI hashes |
| `features-124d.npz` | 124-d residual vectors for Model 2 |
| `probe-weights-clip-l-ft.npz` | Ridge mean, scale, weights, `thr_oi_1pct` |
| `meta-split.json` | Meta train 217, hold-out v2 79, hold-out v3 198 |

Run:

```bash
uv run python scripts/retrain_photo_classify.py --pack DIR --out DIR
```

Seeds are fixed: detector `20260940`, provider `20260956`. Two consecutive
runs must be byte-identical.

Not in the **published** retrain pack: raw photographs, COCO/Open Images/Kodak
binaries, CORD, Danbooru. Those files stay in their public corpora or in the
private freeze archive. Pixel rebuild of a missing embedding still needs
`clip-l-ft.pt` and the original file.

## Where the rows come from

Identity is always sha256 of the file bytes.

### Public, named corpora

| Role | Source |
| --- | --- |
| `detector_photo_train`, `detector_photo_coco_hold` | COCO 2017 validation stills |
| `detector_open_images_*`, `detector_photo_dev_oi` | Open Images photographs. Locked fresh never enters train |
| `detector_photo_kodak` | Kodak lossless true color, 24 files |
| `detector_photo_picsum_eval` | Picsum |
| `detector_flux_hf_hold` | Hugging Face FLUX stills, hold-out |
| Flickr UI cell (150) | Openverse `license_type=all-cc`, host `live.staticflickr.com` only |
| Meta hold-out v3 (198) | Meta Muse Image API, eval-only, not in train |

### In-house labeled photographs, not redistributed

Provider OpenAI, Google, and TC260 train and test rows, and most detector AI
train and test rows, are labeled photographic generations held privately.
The published pack carries only their sha256, roles, and precomputed vectors.
It does not carry the pixels.

### Out of the freeze train

CORD receipts, FUNSD forms, Danbooru, charts, and map tiles were measured as
failure domains. They are not Model 1 train. A future universal router treats
them as other domains, not as extra CLIP-L negatives.

## What the freeze script does

Historically `freeze_both_models.py` built the catalog from local paths, then
trained. That campaign code is private and is not in this repository. The
public entry is `scripts/retrain_photo_classify.py`, which skips catalog
construction and fits the same heads from the pack above.

1. Load CLIP-L-ft vectors by sha256. Fit the 768-512-128-1 MLP on
   `detector_ai_train` plus extras versus `detector_photo_train` plus struct.
   Threshold is the 1.67% quantile on `detector_photo_dev_oi`.
2. AND that MLP with the frozen ridge in the probe file. DEFINITELY is both.
3. Load 124-d vectors. Fit one-vs-rest focal heads for openai, google, bytedance, tc260,
   meta_muse_image, no_ai. Margin 0.30. The public class for the Muse Image
   head is `muse-image`. `tc260` is the China AIGC label standard, mixed
   producers, not one company.
4. Repeat with the same seeds. Compare bytes.

CLIP-L fine-tune of the last two vision blocks is earlier, seed 20260822, and
is not repeated by the retrain script. The backbone file `clip-l-ft.pt` is an
inference and embedding dependency, not a retrain-pack input.

## Next retrain

The taxonomy is GENERATOR-based: a class names the model that rendered the
pixels, never the brand. `bytedance` (the shared Doubao+Jimeng lineage)
shipped 2026-09-07 as a surgical head refit at 83.9% on the frozen test
cell; the rest of China (Qwen, Yuanbao, Kling, ...) abstains behind the
tc260 veto until per-producer train mass holds its own cells (qwen
measured at 64 train rows: 27%). The producer metadata is available
locally (all catalog tc260 rows parse their producer from the
sha256-verified mirror),
and the historical harvest keeps adding mass
(`tc260-producer-split-2026-09-07/historical-harvest.json`). Microsoft
has no pixel class and cannot have one: its products render with other
vendors' models, named by their own C2PA softwareAgent.

The 2026-09-02 drift campaign set two preconditions for adding renderer
drift rows to that fit. First, scale: 14 openai drift rows recovered exactly
one near-miss row in leave-one-out while regressing a sibling cell through
the shared negative pools (google -10 or meta -6 depending on variant), so
drift rows should enter together with enough mass to hold every cell flat,
not as a standalone patch. Second, scope: 6 of 12 Aug rows failed before
the provider stage, so a drift fix that covers the whole cell needs Model 1
in the loop. The `ContentProducer` metadata is already available; the
remaining prerequisite is enough independent per-producer training and
evaluation data to establish a stable class without regressing sibling cells.