mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-08-31 17:50:35 +02:00
Align watermark content with detector coverage
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
"""Consolidate the hand-labelled contact-sheet rounds into ONE ground-truth file.
|
||||
"""Consolidate the hand-labeled contact-sheet rounds into ONE ground-truth file.
|
||||
|
||||
Ground truth is `uid -> the set of visible marks actually present`, hand-labelled
|
||||
Ground truth is `uid -> the set of visible marks actually present`, hand-labeled
|
||||
blind against contact sheets with a two-sided control in every round. Rounds so far:
|
||||
|
||||
2026-07-18 text-mark/pill round : 423 cells (doubao / jimeng / jimeng_pill arms)
|
||||
@@ -10,12 +10,12 @@ DATA SAFETY: treat the input dataset as sensitive. This script reads a gitignore
|
||||
dataset and writes a gitignored ground-truth file. Neither the images nor this
|
||||
output may be committed; only the harness is. See the repo CLAUDE.md.
|
||||
|
||||
The labels record what the LABELLER SAW in the crop, one of:
|
||||
The labels record what the labeler saw in the crop, one of:
|
||||
doubao | jimeng | pill | sparkle | other_ai_label | none | uncertain
|
||||
`other_ai_label` is a real visible AI label from a vendor we do NOT have a mark for
|
||||
(千问 / 百度 / 星绘 / 抖音); it is NOT a positive for any registered mark, but it is
|
||||
also not "clean" -- it is exactly what the relaxed jimeng detector confuses.
|
||||
`uncertain` rows are EXCLUDED from scoring rather than coerced, so a labeller's
|
||||
`uncertain` rows are EXCLUDED from scoring rather than coerced, so a labeler's
|
||||
honest doubt never becomes a fabricated data point.
|
||||
"""
|
||||
|
||||
@@ -32,7 +32,7 @@ SEEN_TO_MARK = {
|
||||
"pill": "jimeng_pill",
|
||||
"sparkle": "gemini",
|
||||
}
|
||||
# Which marks a crop centred on `key` lets the labeller rule on (same corner = visible
|
||||
# Which marks a crop centered on `key` lets the labeler rule on (same corner = visible
|
||||
# in the same crop). Doubao and Jimeng share the bottom-right corner.
|
||||
_ADJUDICATES = {
|
||||
"doubao": ("doubao", "jimeng"),
|
||||
@@ -83,7 +83,7 @@ def main() -> None:
|
||||
type=Path,
|
||||
nargs="?",
|
||||
default=Path(".local-eval/textmark-relaxation"),
|
||||
help="Directory containing the blinded labelling rounds",
|
||||
help="Directory containing the blinded labeling rounds",
|
||||
)
|
||||
root = parser.parse_args().root
|
||||
out = root / "groundtruth.jsonl"
|
||||
@@ -105,8 +105,8 @@ def main() -> None:
|
||||
m["uid"],
|
||||
{"uid": m["uid"], "path": m["path"], "present": [], "seen": [], "rounds": [], "adjudicated": []},
|
||||
)
|
||||
# ADJUDICATION SCOPE -- load-bearing. A crop centred on one mark only lets
|
||||
# the labeller rule on marks visible IN THAT CROP. A pill crop (top-left)
|
||||
# ADJUDICATION SCOPE -- load-bearing. A crop centered on one mark only lets
|
||||
# the labeler rule on marks visible IN THAT CROP. A pill crop (top-left)
|
||||
# says nothing about a bottom-right wordmark, so scoring jimeng against a
|
||||
# pill-round image would book real detections as false fires (~61% of pills
|
||||
# carry a wordmark). Bottom-right marks co-adjudicate each other: one crop
|
||||
|
||||
Reference in New Issue
Block a user