mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-08-09 23:50:40 +02:00
Collapse the duplicated detection path and lift the image pipeline into the library
The visible-mark path had grown three copies of one ladder sweep, four
near-identical `detect` arms, and four hand-rolled `footprint_mask` overrides;
mark knowledge sat in five hand-maintained tables across three modules; and the
flagship `all`/`batch` pipeline existed only in cli.py, written twice with
divergent behavior.
Detection is now one measurement. `_ladder_best` replaces the three sweeps,
`_scan`/`_verdict` replace the four arms, and the winning box travels to the
mask on `TextMarkDetection.match_box` instead of being swept a second time.
`detect_both` returns the strict and relaxed verdicts from one scan, which
halves the arbiter's perception cost (260 -> 130 matchTemplate calls on a 2048²
image, verdicts identical field for field). A per-mark demotion goes in the new
`_post_gate` hook, never in a `detect` override -- an override is invisible to
the single-pass path, which is how the RunningHub and Yuanbao anchor gates
briefly stopped applying.
Everything about a mark is now one registry row: product, label regime, the
platform sentence `identify` reports, the metadata signals that confirm it, and
its TC260 producer codes. `identify._VISIBLE_MARK_PLATFORM`, the signal mapping
in `api.visible_provenance`, `_PRODUCT_OF` and the pill veto are derived from
those rows.
`api.remove_all` / `api.remove_batch` are the library form of the `all` and
`batch` commands; the CLI is a wrapper that owns console text and exit codes.
Progress is a `(stage, detail)` pair of stable tokens, so the CLI keys its
wording off structure rather than parsing the library's prose back.
Two intentional behavior changes, both verified against a recorded 811-image
sample of detector verdicts, removal-mask hashes, arbiter decisions and
`identify` reports:
* A TC260 label now relaxes the vendor its `ContentProducer` names rather than
ByteDance's pair on every China-AIGC image. 333 of 811 samples move; on 185
of them the previously relaxed pair was simply the wrong vendor, and the
mark actually present never reached the relaxed gate its own
`provenance_ncc_factor` was calibrated for.
* A confident LibLibAI detection suppresses the Jimeng pill, like every other
TC260 product's mark. It was registered alongside RunningHub and Baidu, both
of which were added to the hand-written veto list, and it was not. 1 sample
moves, and it is exactly the co-firing case.
Nothing else in that record changes: detector verdicts, mask hashes and
`identify` verdicts are byte-identical, and all 200 calibration constants are
untouched.
Also: `aigc_label` and friends plus `extract_c2pa_info` are memoized on
(path, mtime_ns, size) -- size because this package rewrites in place; the
native TC260 container readers route on magic bytes instead of the file
extension, so a mislabeled AVI or FLV is no longer invisible; `identify` shares
one pixel decode between the DWT-DCT and visible stages (TrustMark keeps its own
Pillow decode, which is not substitutable); and the six `stabilize_*` video
wrappers collapse into one policy table.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
480f478484
commit
78d9e81d0f
@@ -1,7 +1,7 @@
|
||||
"""How much recall is the coarse scale ladder costing, and what would a denser one cost?
|
||||
|
||||
THE FINDING THIS MEASURES
|
||||
`_tophat_score` sweeps three rungs -- (0.8, 1.0, 1.25) -- and the `binary` front-end
|
||||
`_ladder_best` sweeps three rungs -- (0.8, 1.0, 1.25) -- and the `binary` front-end
|
||||
sweeps none at all. Measured on stamped marks over controlled backgrounds
|
||||
(`scripts/detector_response.py`), the response is a COMB: doubao scores 0.99 exactly at
|
||||
each rung and collapses to 0.37-0.48 between them, against a 0.50 gate. So a mark whose
|
||||
@@ -93,7 +93,10 @@ def wilson(k: int, n: int, z: float = 1.96) -> tuple[float, float]:
|
||||
|
||||
|
||||
def score_at_scales(engine: Any, image: np.ndarray, scales: tuple[float, ...]) -> dict[float, float]:
|
||||
"""`_tophat_score`'s inner loop, opened up so the ladder is a parameter.
|
||||
"""`_ladder_best`'s inner loop, opened up so the ladder is a parameter.
|
||||
|
||||
A DELIBERATE second copy of the sweep: it exists to measure ladders the shipped
|
||||
engine does not have, so it must not be folded back into `_ladder_best`.
|
||||
|
||||
Deliberately reaches into the engine (`tophat_response`, `_glyph_silhouette`): a
|
||||
measurement script may, product code may not. Kept a faithful copy of the shipped
|
||||
|
||||
@@ -62,17 +62,10 @@ OUT = REPO / ".local-eval" / "vendor-cohorts.jsonl"
|
||||
FIRED = REPO / ".local-eval" / "visible-positives.jsonl"
|
||||
SHEET_DIR = REPO / ".local-eval" / "vendor-cohort-sheets"
|
||||
|
||||
# A producer code is `001` + `1` + USCC(18) + a 5-digit app/product suffix, so two
|
||||
# codes sharing the USCC are the same legal entity registering different products.
|
||||
# Slicing is defensive: anything not matching the layout is grouped by its raw value.
|
||||
_USCC_START, _USCC_END = 4, 22
|
||||
|
||||
|
||||
def uscc_of(code: str) -> str:
|
||||
"""The 18-char Unified Social Credit Code embedded in a TC260 producer code."""
|
||||
if len(code) >= _USCC_END and code[:3] == "001":
|
||||
return code[_USCC_START:_USCC_END]
|
||||
return code
|
||||
# `uscc_of` moved into the library (`metadata.uscc_of`) when the USCC -> vendor table
|
||||
# started driving `api.visible_provenance`; this script must group by the same rule the
|
||||
# product uses, so it imports rather than reimplements it.
|
||||
from remove_ai_watermarks.metadata import uscc_of # noqa: E402
|
||||
|
||||
|
||||
def _one(path_str: str) -> dict[str, Any] | None:
|
||||
|
||||
@@ -97,7 +97,7 @@ def _score(args: ScoreArgs) -> dict[str, Any] | None:
|
||||
eng = TextMarkEngine(build_config(asset, name, basis, overrides))
|
||||
loc = eng.locate(img)
|
||||
try:
|
||||
score, box = eng._tophat_best(img, loc)
|
||||
score, box = eng._ladder_best(img, loc)
|
||||
except Exception:
|
||||
return None
|
||||
return {"path": path_str, "score": round(float(score), 4), "box": box}
|
||||
@@ -225,7 +225,7 @@ def sheets(pos: list[dict[str, Any]], name: str, per_sheet: int = 24) -> None:
|
||||
# genuinely unknown, which is the one case a dense ladder earns its cost -- but it also
|
||||
# hands clean corners extra chances to match, so it must never set a gate.
|
||||
_FIT_SCALES = tuple(round(0.4 * (1.03**i), 4) for i in range(80)) # 0.40 .. ~4.1
|
||||
# The ladder the product actually ships (`_text_mark_engine._tophat_best`).
|
||||
# The ladder the product actually ships (`_text_mark_engine._ladder_best`).
|
||||
_SHIPPED_LADDER = (0.8, 1.0, 1.25)
|
||||
|
||||
|
||||
@@ -460,7 +460,7 @@ def _cross_score(args: tuple[str, Any, Any]) -> dict[str, Any] | None:
|
||||
eng = TextMarkEngine(cfg)
|
||||
loc = eng.locate(img)
|
||||
try:
|
||||
score, _ = eng._tophat_best(img, loc)
|
||||
score, _ = eng._ladder_best(img, loc)
|
||||
except Exception:
|
||||
return None
|
||||
out[key] = round(float(score), 4)
|
||||
|
||||
Reference in New Issue
Block a user