mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-08-19 03:57:13 +02:00
test(eval): fix the qwen_in pipeline-fidelity eval set + PaddleOCR ground-truth flow
- data/qwen_in/: a stable, committed set of 4 AI-generated images (OpenAI +
Google, carrying SynthID/C2PA -- same class as data/samples fixtures) used to
compare the controlnet/sdxl/qwen pipelines for fidelity. Two text-multi-script
(incl. RU/CJK), one EN poster, one face grid. README documents the set + the
ground-truth workflow. data/ is sdist-excluded so the wheel is unaffected.
- scripts/fidelity_metrics.py: switch text OCR from EasyOCR to PaddleOCR
(PP-OCRv6, higher accuracy esp. CJK, single multilingual stack); split into
`ocr` (seed a {basename: text} ground truth) and `compare` (--ground-truth for
a clean CER vs the hand-verified reference instead of noisy OCR-vs-OCR). Spatial
IoU-NMS keeps the best-scoring read per line so wrong-script models don't inject
garbage over Cyrillic/CJK.
- Oracle methodology: validate the OpenAI arm FIRST (openai.com/verify is more
accessible and the strongest Playwright/Chrome-MCP automation candidate; the
Gemini app is more manual). Recorded in CLAUDE.md + docs/synthid.md.
Ground-truth JSON (data/qwen_in/ground_truth.json) lands in a follow-up once
hand-verified.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
a2c33af284
commit
e29c156279
@@ -181,6 +181,15 @@ pos original plus its minimum-clearing cleaned output (manifest `verified_via` =
|
||||
was oracle-verified but is not committed (third-party content stays out of the
|
||||
public corpus).
|
||||
|
||||
**Oracle validation order: start with OpenAI.** When validating removal across
|
||||
vendors, run the OpenAI arm first. `openai.com/verify` is more accessible than the
|
||||
Gemini app -- fewer per-check restrictions, so it gives the fastest signal and is
|
||||
the strongest candidate for automation (Playwright / Chrome MCP driving
|
||||
`openai.com/verify`); the Gemini "Verify with SynthID" flow is more manual. This is
|
||||
an ordering/throughput choice, not a substitution: each oracle only reads its own
|
||||
vendor's SynthID (`openai.com/verify` is OpenAI-scoped), so Google content still
|
||||
needs the Gemini app.
|
||||
|
||||
| Vendor | Images | Resolution(s) | Pipeline | Removed at |
|
||||
|--------|--------|---------------|----------|------------|
|
||||
| OpenAI (gpt-image) | n=4 (3 archived + 1 external-only) | 1024x1536 .. 1600x1600 | native | **0.05** |
|
||||
|
||||
Reference in New Issue
Block a user