mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-08-07 22:48:37 +02:00
38 lines
1.4 KiB
Markdown
38 lines
1.4 KiB
Markdown
# Pipeline fidelity evaluation
|
|
|
|
This evaluation compares diffusion pipelines with
|
|
`scripts/fidelity_metrics.py`. The images themselves have one canonical home
|
|
in `data/synthid/originals/`; this directory stores only evaluation-specific
|
|
ground truth and instructions.
|
|
|
|
| Original | Provider | Content | Exercises |
|
|
| --- | --- | --- | --- |
|
|
| `ChatGPT Image May 31, 2026, 02_03_55 PM.png` | OpenAI | Multilingual typography | Text preservation |
|
|
| `Gemini_Generated_Image_633uuy633uuy633u.png` | Google | Landscape with a Chinese sign | CJK text preservation |
|
|
| `Gemini_Generated_Image_y48j3cy48j3cy48j.png` | Google | Portrait grid | Face identity and skin texture |
|
|
|
|
## Text ground truth
|
|
|
|
`ground-truth.json` contains hand-verified OCR for the two text-bearing
|
|
originals. To regenerate an OCR seed:
|
|
|
|
```bash
|
|
uv run scripts/fidelity_metrics.py ocr \
|
|
"data/synthid/originals/ChatGPT Image May 31, 2026, 02_03_55 PM.png" \
|
|
data/synthid/originals/Gemini_Generated_Image_633uuy633uuy633u.png \
|
|
--langs en,ru,ch \
|
|
--out data/evaluations/fidelity/ground-truth.json
|
|
```
|
|
|
|
Verify and correct the generated text by hand before using it as ground truth.
|
|
|
|
## Compare
|
|
|
|
```bash
|
|
uv run scripts/fidelity_metrics.py compare \
|
|
--original data/synthid/originals/Gemini_Generated_Image_y48j3cy48j3cy48j.png \
|
|
--variant controlnet=<out>.png \
|
|
--variant qwen=<out>.png \
|
|
--ocr-langs ""
|
|
```
|