mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-08-31 09:40:38 +02:00
Reject the uncalibrated text-manifest tiling and verify Content Seal transforms
Tiled diffusion was never provider-oracle calibrated with verified text restoration: the tiled VAE donor path ran anyway and produced results no oracle had certified. The combination is now rejected at both the pipeline and the engine seam (ValueError with the reason), and the CLI help no longer implies support. The invisible help is generalized and the metadata container list corrected (MKA/OGA/Opus/AAC). scripts/contentseal_transforms.py reproduces the deterministic crop, resize, and JPEG variants of the Content Seal corpus from manifest.csv, hash-verifying every output; its README gains scripts/README.md context and new data tests. The corpus README is honest about the one crop the daily oracle limit left unchecked, and the eval CSVs carry the updated verdicts. The byte-scan SynthID suppression hoists its soft-binding lookup so the guard is computed once. Staged on top of 0.33.1; no version bump in this commit.
This commit is contained in:
@@ -0,0 +1,68 @@
|
||||
# Maintainer scripts
|
||||
|
||||
These scripts are development and evaluation tools, not installed CLI commands. Run
|
||||
them from the repository root with `uv run python scripts/<name>.py --help`. Inputs
|
||||
under `.local-eval/` and generated reports remain untracked unless a data README
|
||||
explicitly names a tracked canonical result.
|
||||
|
||||
## Audits and release checks
|
||||
|
||||
| Script | Purpose |
|
||||
| --- | --- |
|
||||
| `corpus_gap_scan.py` | Compare a local image corpus with the library's `identify` results. |
|
||||
| `detection_timing.py` | Record per-method metadata and verdict timings. |
|
||||
| `detection_timing_report.py` | Aggregate timing records by method and segment. |
|
||||
| `fidelity_metrics.py` | Compute objective image-fidelity metrics for paired outputs. |
|
||||
| `invisible_quality_audit.py` | Pair originals and invisible-removal outputs for quality review. |
|
||||
| `metadata_removal_audit.py` | Check metadata detection/removal parity over a corpus. |
|
||||
| `pill_gate_audit.py` | Measure the Jimeng pill detector on the product path. |
|
||||
| `real_examples_e2e.py` | Run end-to-end confidence checks over local examples. |
|
||||
| `record_parity_audit.py` | Compare record-based and file-based identification. |
|
||||
| `resource_ceilings.py` | Measure peak RSS and runtime for fill backends. |
|
||||
| `robustness_suite.py` | Exercise CLI failures on adversarial and degenerate inputs. |
|
||||
| `sidecar_regression.py` | Compare current identification with recorded sidecars. |
|
||||
| `smoke_matrix.py` | Exercise CLI parameter choices on real local data. |
|
||||
| `video_fidelity_probe.py` | Compare delivered video fidelity with its source. |
|
||||
| `visible_eval.py` | Benchmark registered visible-mark detectors. |
|
||||
| `visible_removal_audit.py` | Audit visible-removal results over a local corpus. |
|
||||
|
||||
## Calibration and corpus preparation
|
||||
|
||||
| Script | Purpose |
|
||||
| --- | --- |
|
||||
| `contentseal_transforms.py` | Reproduce and hash-check deterministic Content Seal variants. |
|
||||
| `detector_response.py` | Measure detector response over mark size, contrast, background, and aspect. |
|
||||
| `fill_quality.py` | Measure visible-fill quality against constructed ground truth. |
|
||||
| `ladder_headroom.py` | Measure recall cost from the coarse scale ladder. |
|
||||
| `synthid_corpus.py` | Ingest and inspect the local SynthID reference corpus. |
|
||||
| `vendor_cohort_harvest.py` | Partition TC260 carriers by producer code. |
|
||||
| `vendor_mark_calibrate.py` | Calibrate a candidate vendor text detector. |
|
||||
| `visible_alpha_solve.py` | Rebuild visible-watermark alpha assets from controlled captures. |
|
||||
| `visible_groundtruth.py` | Consolidate blinded contact-sheet labels into ground truth. |
|
||||
| `visible_positives.py` | List corpus images carrying a registered visible mark. |
|
||||
| `visible_recall_sample.py` | Build an unbiased blinded sample for recall measurement. |
|
||||
| `visible_sheets.py` | Build blinded contact sheets for relaxation candidates. |
|
||||
|
||||
## Research and diagnostic prototypes
|
||||
|
||||
| Script | Purpose |
|
||||
| --- | --- |
|
||||
| `cjk_tail_probe.py` | Test a generic template for otherwise uncovered CJK labels. |
|
||||
| `controlnet_sweep.py` | Sweep the historical ControlNet removal prototype. |
|
||||
| `infer_text_lines.py` | Draft stable source-text lines without modifying pixels. |
|
||||
| `qwen_scrub_prototype.py` | Probe low-strength Qwen regeneration on a GPU. |
|
||||
| `selective_text_restoration.py` | Evaluate text restoration over a scrubbed image. |
|
||||
| `synthid_pixel_probe.py` | Run the experimental local SynthID carrier probe. |
|
||||
| `video_synthid_sweep.py` | Build oracle-gated video regeneration candidates. |
|
||||
|
||||
## Generated assets
|
||||
|
||||
| Script | Purpose |
|
||||
| --- | --- |
|
||||
| `render_pill_silhouette.py` | Render the synthetic Jimeng pill silhouette. |
|
||||
| `render_vendor_silhouettes.py` | Render synthetic vendor text-mark silhouettes. |
|
||||
|
||||
## Shared helpers
|
||||
|
||||
`_plain_console.py` provides plain-text fallbacks for Rich output, and
|
||||
`_text_eval.py` contains normalization helpers shared by text-evaluation scripts.
|
||||
@@ -0,0 +1,82 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Reproduce the deterministic Content Seal crop, resize, and JPEG variants."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import hashlib
|
||||
import logging
|
||||
from pathlib import Path
|
||||
|
||||
from PIL import Image
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
CORPUS = ROOT / "data" / "contentseal"
|
||||
MANIFEST = CORPUS / "manifest.csv"
|
||||
|
||||
|
||||
def _rows() -> dict[str, dict[str, str]]:
|
||||
with MANIFEST.open(newline="", encoding="utf-8") as stream:
|
||||
return {row["name"]: row for row in csv.DictReader(stream)}
|
||||
|
||||
|
||||
def _write_and_verify(image: Image.Image, path: Path, row: dict[str, str], *, format: str, quality: int) -> None:
|
||||
image.save(path, format=format, quality=quality)
|
||||
digest = hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
if digest != row["sha256"]:
|
||||
raise RuntimeError(f"{row['name']} hash mismatch: expected {row['sha256']}, got {digest}")
|
||||
log.info("Verified %s", path)
|
||||
|
||||
|
||||
def reproduce_transforms(output_dir: Path) -> list[Path]:
|
||||
"""Write and hash-check the eight deterministic manifest variants."""
|
||||
rows = _rows()
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
outputs: list[Path] = []
|
||||
|
||||
for prefix, source_name in (("fox", "gen_fox_forest"), ("text", "gen_text_poster")):
|
||||
with Image.open(CORPUS / rows[source_name]["file"]) as opened:
|
||||
source = opened.convert("RGB")
|
||||
|
||||
for fraction in (0.5, 0.33):
|
||||
width = int(source.width * fraction)
|
||||
height = int(source.height * fraction)
|
||||
left = (source.width - width) // 2
|
||||
top = (source.height - height) // 2
|
||||
name = f"{prefix}_crop{int(fraction * 100)}"
|
||||
path = output_dir / f"{name}.webp"
|
||||
crop = source.crop((left, top, left + width, top + height))
|
||||
_write_and_verify(crop, path, rows[name], format="WEBP", quality=95)
|
||||
outputs.append(path)
|
||||
|
||||
scale = 512 / max(source.size)
|
||||
resized = source.resize(
|
||||
(round(source.width * scale), round(source.height * scale)),
|
||||
Image.Resampling.LANCZOS,
|
||||
)
|
||||
name = f"{prefix}_res512"
|
||||
path = output_dir / f"{name}.webp"
|
||||
_write_and_verify(resized, path, rows[name], format="WEBP", quality=95)
|
||||
outputs.append(path)
|
||||
|
||||
name = f"{prefix}_jpeg85"
|
||||
path = output_dir / f"{name}.jpg"
|
||||
_write_and_verify(source, path, rows[name], format="JPEG", quality=85)
|
||||
outputs.append(path)
|
||||
|
||||
return outputs
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("output_dir", type=Path, help="Directory for regenerated variants")
|
||||
args = parser.parse_args()
|
||||
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
|
||||
reproduce_transforms(args.output_dir)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -21,9 +21,9 @@ honest doubt never becomes a fabricated data point.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
SEEN_TO_MARK = {
|
||||
@@ -77,7 +77,15 @@ def metadata_provenance(path: str) -> list[str]:
|
||||
|
||||
|
||||
def main() -> None:
|
||||
root = Path(sys.argv[1] if len(sys.argv) > 1 else ".local-eval/textmark-relaxation")
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument(
|
||||
"root",
|
||||
type=Path,
|
||||
nargs="?",
|
||||
default=Path(".local-eval/textmark-relaxation"),
|
||||
help="Directory containing the blinded labelling rounds",
|
||||
)
|
||||
root = parser.parse_args().root
|
||||
out = root / "groundtruth.jsonl"
|
||||
rows: dict[str, dict] = {}
|
||||
stats: dict[str, int] = {}
|
||||
|
||||
@@ -22,10 +22,10 @@ Design decisions that matter:
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import json
|
||||
import random
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
@@ -35,6 +35,8 @@ import numpy as np
|
||||
if TYPE_CHECKING:
|
||||
from numpy.typing import NDArray
|
||||
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
|
||||
from remove_ai_watermarks.image_io import imread
|
||||
|
||||
@@ -63,10 +65,16 @@ def corner_strip(img: NDArray[Any]) -> NDArray[Any] | None:
|
||||
|
||||
|
||||
def main() -> None:
|
||||
scan = Path(sys.argv[1])
|
||||
out = Path(sys.argv[2])
|
||||
n_tc260 = int(sys.argv[3]) if len(sys.argv) > 3 else 160
|
||||
n_google = int(sys.argv[4]) if len(sys.argv) > 4 else 80
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("scan", type=Path, help="JSONL corpus scan produced by the visible evaluation harness")
|
||||
parser.add_argument("output", type=Path, help="Directory for contact sheets and the blinded manifest")
|
||||
parser.add_argument("--tc260", type=int, default=160, help="Number of TC260 carriers to sample")
|
||||
parser.add_argument("--google", type=int, default=80, help="Number of Google-provenance carriers to sample")
|
||||
args = parser.parse_args()
|
||||
scan = args.scan
|
||||
out = args.output
|
||||
n_tc260 = args.tc260
|
||||
n_google = args.google
|
||||
out.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
recs = [json.loads(line) for line in scan.open() if '"marks"' in line]
|
||||
|
||||
@@ -12,6 +12,7 @@ Each sheet mixes three strata in shuffled order:
|
||||
The two control strata are what make a low measured precision trustworthy.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import json
|
||||
import random
|
||||
@@ -62,9 +63,13 @@ def crop(path: str, region: tuple[int, int, int, int] | None, pad_factor: float
|
||||
|
||||
|
||||
def main() -> None:
|
||||
with open(sys.argv[1]) as fh:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("input", type=Path, help="JSON candidate list")
|
||||
parser.add_argument("output", type=Path, help="Directory for blinded contact sheets")
|
||||
args = parser.parse_args()
|
||||
with args.input.open() as fh:
|
||||
items = json.load(fh) # [{uid,path,key,stratum,conf}]
|
||||
outdir = Path(sys.argv[2])
|
||||
outdir = args.output
|
||||
outdir.mkdir(parents=True, exist_ok=True)
|
||||
random.Random(1234).shuffle(items) # noqa: S311 -- sheet ordering, not cryptography
|
||||
|
||||
|
||||
Reference in New Issue
Block a user