test(eval): fix the qwen_in pipeline-fidelity eval set + PaddleOCR ground-truth flow

- data/qwen_in/: a stable, committed set of 4 AI-generated images (OpenAI +
  Google, carrying SynthID/C2PA -- same class as data/samples fixtures) used to
  compare the controlnet/sdxl/qwen pipelines for fidelity. Two text-multi-script
  (incl. RU/CJK), one EN poster, one face grid. README documents the set + the
  ground-truth workflow. data/ is sdist-excluded so the wheel is unaffected.
- scripts/fidelity_metrics.py: switch text OCR from EasyOCR to PaddleOCR
  (PP-OCRv6, higher accuracy esp. CJK, single multilingual stack); split into
  `ocr` (seed a {basename: text} ground truth) and `compare` (--ground-truth for
  a clean CER vs the hand-verified reference instead of noisy OCR-vs-OCR). Spatial
  IoU-NMS keeps the best-scoring read per line so wrong-script models don't inject
  garbage over Cyrillic/CJK.
- Oracle methodology: validate the OpenAI arm FIRST (openai.com/verify is more
  accessible and the strongest Playwright/Chrome-MCP automation candidate; the
  Gemini app is more manual). Recorded in CLAUDE.md + docs/synthid.md.

Ground-truth JSON (data/qwen_in/ground_truth.json) lands in a follow-up once
hand-verified.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Victor Kuznetsov
2026-06-20 14:17:04 -07:00
co-authored by Claude Opus 4.8
parent a2c33af284
commit e29c156279
8 changed files with 194 additions and 73 deletions
+9
View File
@@ -181,6 +181,15 @@ pos original plus its minimum-clearing cleaned output (manifest `verified_via` =
was oracle-verified but is not committed (third-party content stays out of the
public corpus).
**Oracle validation order: start with OpenAI.** When validating removal across
vendors, run the OpenAI arm first. `openai.com/verify` is more accessible than the
Gemini app -- fewer per-check restrictions, so it gives the fastest signal and is
the strongest candidate for automation (Playwright / Chrome MCP driving
`openai.com/verify`); the Gemini "Verify with SynthID" flow is more manual. This is
an ordering/throughput choice, not a substitution: each oracle only reads its own
vendor's SynthID (`openai.com/verify` is OpenAI-scoped), so Google content still
needs the Gemini app.
| Vendor | Images | Resolution(s) | Pipeline | Removed at |
|--------|--------|---------------|----------|------------|
| OpenAI (gpt-image) | n=4 (3 archived + 1 external-only) | 1024x1536 .. 1600x1600 | native | **0.05** |