mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-08-09 23:50:40 +02:00
Collapse the duplicated detection path and lift the image pipeline into the library
The visible-mark path had grown three copies of one ladder sweep, four
near-identical `detect` arms, and four hand-rolled `footprint_mask` overrides;
mark knowledge sat in five hand-maintained tables across three modules; and the
flagship `all`/`batch` pipeline existed only in cli.py, written twice with
divergent behavior.
Detection is now one measurement. `_ladder_best` replaces the three sweeps,
`_scan`/`_verdict` replace the four arms, and the winning box travels to the
mask on `TextMarkDetection.match_box` instead of being swept a second time.
`detect_both` returns the strict and relaxed verdicts from one scan, which
halves the arbiter's perception cost (260 -> 130 matchTemplate calls on a 2048²
image, verdicts identical field for field). A per-mark demotion goes in the new
`_post_gate` hook, never in a `detect` override -- an override is invisible to
the single-pass path, which is how the RunningHub and Yuanbao anchor gates
briefly stopped applying.
Everything about a mark is now one registry row: product, label regime, the
platform sentence `identify` reports, the metadata signals that confirm it, and
its TC260 producer codes. `identify._VISIBLE_MARK_PLATFORM`, the signal mapping
in `api.visible_provenance`, `_PRODUCT_OF` and the pill veto are derived from
those rows.
`api.remove_all` / `api.remove_batch` are the library form of the `all` and
`batch` commands; the CLI is a wrapper that owns console text and exit codes.
Progress is a `(stage, detail)` pair of stable tokens, so the CLI keys its
wording off structure rather than parsing the library's prose back.
Two intentional behavior changes, both verified against a recorded 811-image
sample of detector verdicts, removal-mask hashes, arbiter decisions and
`identify` reports:
* A TC260 label now relaxes the vendor its `ContentProducer` names rather than
ByteDance's pair on every China-AIGC image. 333 of 811 samples move; on 185
of them the previously relaxed pair was simply the wrong vendor, and the
mark actually present never reached the relaxed gate its own
`provenance_ncc_factor` was calibrated for.
* A confident LibLibAI detection suppresses the Jimeng pill, like every other
TC260 product's mark. It was registered alongside RunningHub and Baidu, both
of which were added to the hand-written veto list, and it was not. 1 sample
moves, and it is exactly the co-firing case.
Nothing else in that record changes: detector verdicts, mask hashes and
`identify` verdicts are byte-identical, and all 200 calibration constants are
untouched.
Also: `aigc_label` and friends plus `extract_c2pa_info` are memoized on
(path, mtime_ns, size) -- size because this package rewrites in place; the
native TC260 container readers route on magic bytes instead of the file
extension, so a mislabeled AVI or FLV is no longer invisible; `identify` shares
one pixel decode between the DWT-DCT and visible stages (TrustMark keeps its own
Pillow decode, which is not substitutable); and the six `stabilize_*` video
wrappers collapse into one policy table.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
480f478484
commit
78d9e81d0f
@@ -40,4 +40,23 @@ Do not classify an entire module as untestable because its main path downloads a
|
||||
|
||||
Use availability checks only for paths that actually load large models.
|
||||
|
||||
## One measurement, one gate seam
|
||||
|
||||
A detector is split into a trust-level-blind scan and a verdict that applies the
|
||||
threshold, so `detect` and `detect_both` reach the same numbers by construction. Two
|
||||
rules follow, and both were broken in practice before they were written down:
|
||||
|
||||
- A per-mark demotion goes in the `_post_gate` hook (or, for a whole-scan precondition
|
||||
like LibLibAI's size floor, in `_scan`) -- never in a `detect` override. An override
|
||||
is invisible to `detect_both`, so the RunningHub and Yuanbao anchor gates silently
|
||||
stopped applying on the arbiter's perception path. `TestSinglePassPerception` is the
|
||||
guard: it asserts `detect_both` equals two `detect` calls field for field.
|
||||
- Detection and the removal mask must read ONE sweep. The winning box travels on
|
||||
`TextMarkDetection.match_box` and the registry threads the detection into the mask
|
||||
builder; a mask path that re-runs its own sweep is how the two drift apart.
|
||||
|
||||
Before changing anything in the detection path, record the detectors' exact verdicts
|
||||
over a local sample first and diff them after. A refactor here is only correct if that
|
||||
record is byte-identical, and a green test suite does not establish that on its own.
|
||||
|
||||
Environment setup, dependency recovery, CI behavior, and fixture policy: [`../../docs/development.md`](../../docs/development.md).
|
||||
|
||||
@@ -53,8 +53,5 @@ data/samsung_capture/captures/samsung_content_*
|
||||
# dir on first use). Runtime artifact, never committed.
|
||||
gfpgan/
|
||||
|
||||
# Qwen ControlNet experiment outputs (throwaway eval; never the committed corpus)
|
||||
scripts/_qwen_exp_out/
|
||||
|
||||
# Local-only working data for analysis (not a committed corpus; never tracked)
|
||||
.local-eval/
|
||||
|
||||
@@ -51,4 +51,4 @@ Topic-specific rules live in `.claude/rules/*.md` and are auto-loaded when match
|
||||
|
||||
| File | Covers |
|
||||
|---|---|
|
||||
| `development.md` | Command contracts, project gate, typing boundaries, and model-adjacent tests |
|
||||
| `development.md` | Command contracts, project gate, typing boundaries, model-adjacent tests, and the detection-path measurement rule |
|
||||
|
||||
@@ -128,6 +128,12 @@ remove-ai-watermarks erase image.png \
|
||||
`erase` accepts `cv2`, `migan`, and `lama`. The corresponding optional extra
|
||||
must be installed for a learned backend.
|
||||
|
||||
Two more knobs tune the fill. `--dilate N` (default 3) grows every box by `N`
|
||||
pixels before inpainting, which helps when a mark has a soft edge or a drop
|
||||
shadow just outside the box you measured; it applies to every backend because it
|
||||
shapes the mask. `--inpaint-method telea|ns` selects the classical algorithm and
|
||||
only affects the `cv2` backend.
|
||||
|
||||
## Strip AI metadata
|
||||
|
||||
Inspect metadata:
|
||||
|
||||
@@ -253,9 +253,13 @@ takes the first stable result in specificity order (`sora`, `veo`, `seedance`,
|
||||
candidate. Removal also collects authoritative per-frame timestamps for the
|
||||
encoder, while identification omits that unused ffprobe pass.
|
||||
|
||||
Every per-frame result is untrusted. The provider-specific stabilization
|
||||
wrappers share one recurrence implementation, while retaining separate visual
|
||||
floors and minimum-run policy. Provenance can relax a low-contrast run only
|
||||
Every per-frame result is untrusted. Each provider's floors, minimum-run policy,
|
||||
fill padding and mask style are one row in `VISIBLE_MARK_POLICIES`, and every mark
|
||||
enters the same `stabilize_localizations` entry point; the recurrence
|
||||
implementation underneath knows nothing about providers. That policy row also
|
||||
carries `accepts_provenance`, which forces `provenance=False` for Hailuo and Kling
|
||||
— they have no metadata that could confirm them, and the guarantee used to be
|
||||
structural (their wrappers took no `provenance` parameter at all). Provenance can relax a low-contrast run only
|
||||
after recurring visual evidence exists. Sora transition frames follow the
|
||||
nearest confirmed moving position only with Sora provenance. Veo, Seedance,
|
||||
Dola, Hailuo, and Kling additionally require candidates to remain anchored to
|
||||
@@ -355,6 +359,43 @@ metadata extraction from verdict logic:
|
||||
- `identify` preserves the path-based API and adds the optional registered
|
||||
visible-mark and open invisible-watermark decoders after extraction.
|
||||
|
||||
The DWT-DCT detector and the visible-mark stage share a single decode of the
|
||||
source, held by
|
||||
a per-call `_SharedDecode`. It exposes two accessors because the two arms need
|
||||
opposite failure handling: the visible arm swallows a decode failure (no cv2, no
|
||||
visible marks, metadata verdict untouched), while the invisible arm re-raises it
|
||||
so `has_invisible_target` reaches its documented fail-safe `True`. Swallowing it
|
||||
there would skip a diffusion scrub on a file that used to get one. TrustMark
|
||||
deliberately keeps its own Pillow decode: cv2 and Pillow disagree on EXIF
|
||||
orientation and on 16-bit PNG, so substituting one for the other is not
|
||||
behavior-preserving.
|
||||
|
||||
The metadata probes (`aigc_label`, `xai_signature`, `iptc_ai_system`,
|
||||
`huggingface_job`, `samsung_genai`) and `extract_c2pa_info` are memoized on
|
||||
`(path, mtime_ns, size)`. One `identify` reaches each of them twice, and each
|
||||
re-walks the container or re-runs the manifest reader. Size is in the key as well
|
||||
as mtime because this package rewrites files in place, and an in-place rewrite
|
||||
can land inside one mtime tick. The C2PA key additionally carries the
|
||||
reader-availability flag: with the official reader the manifest comes back as a
|
||||
store and without it from the PNG chunk parser, so the answer depends on process
|
||||
state and not on the file alone.
|
||||
|
||||
Native-container TC260 readers (`isobmff`, `ebml`, `riff`, `flv`) all run, in that
|
||||
order, on every file. Each self-gates on its own magic bytes after a 4-12 byte
|
||||
read, so gating the AVI and FLV ones on the file extension as well was redundant
|
||||
and made a correctly formatted container served under the wrong name invisible.
|
||||
WebP is the one input class the now-unconditional RIFF reader newly touches; its
|
||||
`AVI ` form check is what rejects it.
|
||||
|
||||
`api._SourceEvidence` extracts that metadata once per `remove_all` call and serves
|
||||
both the visible pass (which vendor is confirmed) and the scrub gate (is there an
|
||||
invisible target). It is per-call, never module-level: `batch` may write its
|
||||
output over its input, and a holder that outlived one call would answer the scrub
|
||||
gate from pre-write evidence. In `batch` each stage builds its own holder after
|
||||
any write that precedes it, for the same reason. Every accessor fails safe the way
|
||||
the function it replaces does — no provenance means no relaxation, and an unknown
|
||||
invisible target means scrub rather than skip.
|
||||
|
||||
The `detect` extra composes the shared `pixels` runtime with PyWavelets. Its
|
||||
in-tree [`dwt_dct.py`](../src/remove_ai_watermarks/dwt_dct.py) decoder preserves
|
||||
the upstream matrix algorithm without installing Torch or non-headless OpenCV.
|
||||
@@ -397,6 +438,30 @@ The Jimeng pill has an additional decision gate because its visual detector is
|
||||
weaker than the other registered marks. Keep that policy in the registry, not
|
||||
inside unrelated detector engines.
|
||||
|
||||
Everything about a mark is one registry row: its product family, its label regime,
|
||||
the platform sentence `identify` reports for it, and the metadata signals that
|
||||
confirm its vendor. `identify._VISIBLE_MARK_PLATFORM` and the signal mapping in
|
||||
`api.visible_provenance` are derived from those rows rather than hand-maintained
|
||||
beside them, so registering a mark is one edit. Two marks carry no platform of
|
||||
their own: the Gemini sparkle has its own higher-confidence path, and the
|
||||
capture-less pill is too weak to attribute.
|
||||
|
||||
The set of marks that veto the pill is DERIVED from the registry rows: every mark
|
||||
under the same label regime (`tc260`) belonging to a different product. It used to
|
||||
be a hand-written list of keys, and that list drifted -- LibLibAI was registered
|
||||
alongside RunningHub and Baidu, both of which were added to it, and LibLibAI was
|
||||
not, so a confident LibLibAI detection did not suppress the pill the way its two
|
||||
siblings did. Marks outside the TC260 regime (Gemini, Samsung) are deliberately
|
||||
not vetoers: neither can put `jimeng` into `provenance`, so neither can enable the
|
||||
arm it would be vetoing.
|
||||
|
||||
A TC260 label relaxes the vendor its `ContentProducer` names, resolved through
|
||||
`KnownMark.tc260_producer_codes`. The label itself is vendor-agnostic, so this used to
|
||||
relax ByteDance's two products on every China-AIGC image -- which both risked a
|
||||
false fill on an image carrying some other vendor's mark and denied that vendor's
|
||||
own mark the relaxed gate its `provenance_ncc_factor` was calibrated for. An
|
||||
absent or unmapped producer still falls back to the ByteDance pair.
|
||||
|
||||
`remove_auto_marks` removes every selected mark, not only the strongest one.
|
||||
This matters for images that carry marks in more than one corner.
|
||||
|
||||
@@ -440,7 +505,23 @@ be represented by the shared base:
|
||||
|
||||
The detector and removal mask must use compatible geometry. A detector that
|
||||
fires while producing an empty or misplaced mask is a removal failure even if
|
||||
the detection test passes.
|
||||
the detection test passes. That parity is now structural rather than a
|
||||
convention: the three continuous front ends share one ladder sweep
|
||||
(`_ladder_best`), and the winning box travels to the mask on
|
||||
`TextMarkDetection.match_box` instead of being swept a second time.
|
||||
|
||||
Detection is split into a trust-level-blind `_scan` and a `_verdict` that applies
|
||||
the threshold. `detect_both` returns the strict and relaxed verdicts from one
|
||||
scan, which is what the arbiter's perception stage calls. A per-mark demotion
|
||||
belongs in the `_post_gate` hook, never in a `detect` override: an override is
|
||||
invisible to the single-pass path, and the RunningHub and Yuanbao anchor gates
|
||||
were briefly skipped there for exactly that reason.
|
||||
|
||||
A mark whose removable footprint differs from what the detector localizes
|
||||
overrides `_footprint_rect` (which policy) and `_extend_match_box` (how far the
|
||||
box grows), not the whole `footprint_mask`. Baidu extends right to the corner tag
|
||||
and LibLibAI extends left to the triangle logo; both inherit every guard around
|
||||
that arithmetic.
|
||||
|
||||
Yuanbao uses the polarity-independent `contrast` front end because its standard
|
||||
two-line mark can be light on dark scenes or dark on light scenes. Its detector
|
||||
|
||||
+65
-3
@@ -77,6 +77,55 @@ image = cv2.imread("input.png")
|
||||
result, removed = raiw.remove_visible(image, backend="cv2")
|
||||
```
|
||||
|
||||
## Run the full pipeline
|
||||
|
||||
`remove_all` is the library form of the `all` command: visible marks, then the
|
||||
invisible watermark, then AI metadata. Stages are chained through a file in the
|
||||
system temp directory, so a partial result never appears at the output path.
|
||||
|
||||
```python
|
||||
import remove_ai_watermarks as raiw
|
||||
|
||||
result = raiw.remove_all("input.png", "clean.png") # -> RemoveAllResult
|
||||
print(result.output) # the path written
|
||||
print(result.visible_label) # the marks removed, or None
|
||||
print(result.invisible) # "removed" | "no-signal" | "unavailable"
|
||||
```
|
||||
|
||||
`invisible` is the field to check. `"unavailable"` means the GPU extra is not
|
||||
installed, so the output *looks* processed but still carries the watermark;
|
||||
`"no-signal"` means the scrub was deliberately skipped because nothing was
|
||||
locally detectable, which is a successful run.
|
||||
|
||||
Pass `InvisibleOptions` to tune the diffusion stage, and `engine` to reuse one
|
||||
loaded model across many calls:
|
||||
|
||||
```python
|
||||
from remove_ai_watermarks import InvisibleOptions
|
||||
|
||||
raiw.remove_all(
|
||||
"input.png",
|
||||
"clean.png",
|
||||
invisible=InvisibleOptions(strength=0.35, force=True),
|
||||
progress=print,
|
||||
)
|
||||
```
|
||||
|
||||
If AI metadata survives the strip, `remove_all` raises `MetadataStripIncomplete`
|
||||
**before** writing anything: an AI-readable output is worse than no output.
|
||||
|
||||
`remove_batch` runs one mode over a directory and never lets a single bad file
|
||||
end the run:
|
||||
|
||||
```python
|
||||
summary = raiw.remove_batch("in_dir", "out_dir", mode="visible") # -> BatchSummary
|
||||
print(summary.processed, summary.failed, summary.errors)
|
||||
print(summary.invisible_unavailable) # outputs that still carry the watermark
|
||||
```
|
||||
|
||||
`mode` is `all`, `visible`, `invisible`, or `metadata`. Pass a constructed
|
||||
`InvisibleEngine` as `engine` to load the model once for the whole directory.
|
||||
|
||||
## Inspect provenance
|
||||
|
||||
The default installation evaluates file metadata. Add `visible`, `detect`, or
|
||||
@@ -151,9 +200,22 @@ describe the collector rather than the source file. Pass a C2PA manifest-store
|
||||
dictionary in `record["c2pa_store"]`, or through the explicit
|
||||
`c2pa_manifest_store` argument.
|
||||
|
||||
`identify_from_evidence` does not reopen the source file. It evaluates metadata
|
||||
only; registered visible marks and pixel-backed invisible watermarks remain in
|
||||
the path-based `identify` call.
|
||||
`identify_from_evidence` does not reopen the source file by default: it evaluates
|
||||
metadata only, and registered visible marks and pixel-backed invisible watermarks
|
||||
remain in the path-based `identify` call.
|
||||
|
||||
Pass `image_path` together with `check_visible` or `check_invisible` to add those
|
||||
pixel detectors on top of the SAME evidence. That is how a caller asking one file
|
||||
two provenance questions — which vendor is confirmed, and is there an invisible
|
||||
target — pays for the metadata extraction once:
|
||||
|
||||
```python
|
||||
from remove_ai_watermarks.identify import extract_provenance_evidence, identify_from_evidence
|
||||
|
||||
evidence = extract_provenance_evidence(source)
|
||||
metadata_only = identify_from_evidence(evidence)
|
||||
with_pixels = identify_from_evidence(evidence, image_path=source, check_invisible=True)
|
||||
```
|
||||
|
||||
## Strip metadata
|
||||
|
||||
|
||||
@@ -36,7 +36,7 @@ AI generator values while retaining unrelated camera and editor EXIF. The
|
||||
shared `xai_signature_pair` helper is the single source of truth for the
|
||||
pair. On the ISOBMFF path, `blank_ai_exif_tokens` provides the corresponding
|
||||
in-place scrub for supported EXIF values, TC260 AIGC blocks, and the xAI pair.
|
||||
- **China TC260 AIGC label (caught by `AIGC_MARKERS` / `metadata.aigc_label`, surfaced by `identify` as the `aigc` signal):** China-served generators embed an XMP `<TC260:AIGC>{"Label":"1","ContentProducer":...}` block — China's mandatory AI-content labeling (TC260 namespace `tc260.org.cn/ns/AIGC`).
|
||||
- **China TC260 AIGC label (caught by `AIGC_MARKERS` / `metadata.aigc_label`, surfaced by `identify` as the `aigc` signal):** China-served generators embed an XMP `<TC260:AIGC>{"Label":"1","ContentProducer":...}` block — China's mandatory AI-content labeling (TC260 namespace `tc260.org.cn/ns/AIGC`). The label says only "this is AI", but its `ContentProducer` names the signing entity — `001` + `1` + an 18-char Unified Social Credit Code + a 5-digit product suffix, normalized by `metadata.uscc_of`, or for a few generators a bare product name. `KnownMark.tc260_producer_codes` maps the codes settled per vendor by `scripts/vendor_cohort_harvest.py` to registry mark keys, so an AIGC image relaxes the detector of the vendor it actually carries rather than a guessed pair; an unmapped or absent producer falls back to ByteDance's two products. A code identifies a legal entity, not necessarily one brand, so a hosting or aggregating platform that signs for several apps is a recall bet rather than a proof.
|
||||
|
||||
**Doubao** (ByteDance) uses it (verified on a public issue sample; `ContentProducer` `001191110102MACQD9K64010000`, no C2PA/SynthID/imwatermark — the XMP block is the only signal; GitHub attachment upload did NOT strip it). The same standard is mandatory for Jimeng/Kling/Qwen/Ernie etc., so the one marker covers the whole China-AIGC-labeled ecosystem. `aigc_label` reads **four image serializations** through a shared `_parse` helper: the HTML-entity-encoded XMP `TC260:AIGC` block in **either RDF form** — the nested element `<TC260:AIGC>{...}</TC260:AIGC>` (Doubao) or the attribute `TC260:AIGC="{...}"` (**PicWish**, `ContentProducer="picwish"`, verified on compatible samples) — via a container-agnostic raw-byte scan (any JSON object accepted), a raw-JSON PNG `AIGC` tEXt chunk (Doubao also writes the label this way, no namespaced marker at all — confirmed on compatible samples, `ContentProducer="doubao"`), a bare raw-JSON `{"AIGC":{...}}` object embedded in **JPEG EXIF (UserComment)** by some China-served generators, brace-matched from the scan head with `json.JSONDecoder().raw_decode` (no namespaced marker, no PNG chunk — confirmed on compatible samples, `ContentProducer="001191440300708461136T1308L"`), **and** a bare `AIGC{...}` blob (the label glued straight to its JSON, no `"AIGC":` key wrapper) embedded in a **JPEG APP segment near the JFIF header** — confirmed on compatible samples. The two raw-JSON forms are scanned in one loop (`'"AIGC"'` then `AIGC{`) that **falls through on a non-TC260 / undecodable hit instead of returning** — a quoted `"AIGC"` can appear later in an XMP packet while the real label is a bare `AIGC{...}` earlier in the file, so an unconditional early return on the quoted form would shadow the bare form (the exact bug behind the 06-10 misses). Native MP4/MOV is a fifth serialization: TC260-PG-20257A stores an `AIGC` key in `moov.udta.meta.keys` and the raw JSON in the matching `ilst` item. The seeking parser reaches a tail `moov` without reading `mdat`; removal replaces the key with `free` and blanks the validated value at the same length so every box size and stream offset stays fixed. All generic forms are gated on at least one TC260 field (`TC260_AIGC_FIELDS`) so a generic `AIGC` key cannot false-positive; the namespaced XMP element is unambiguous and needs no gate. `TC260_AIGC_FIELDS` covers **two schemas**: the producer-side one (`Label` / `ContentProducer` / `ProduceID` / `ContentPropagator` / `PropagateID`, Doubao and most China gens) and the **service-provider** one (`ServiceProvider` / `ServiceUser`, plus generic `Time` / `ContentId` which are NOT gated on) — **Tencent Cloud's** AIGC variant (`ServiceProvider` = `腾讯云`), embedded in **EXIF `ImageDescription`**, verified on compatible samples. In `identify`, `aigc` fires on the parsed label **or** the `AIGC_MARKERS` byte scan (the latter preserves the laundering-tell case where the JSON payload is truncated).
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""How much recall is the coarse scale ladder costing, and what would a denser one cost?
|
||||
|
||||
THE FINDING THIS MEASURES
|
||||
`_tophat_score` sweeps three rungs -- (0.8, 1.0, 1.25) -- and the `binary` front-end
|
||||
`_ladder_best` sweeps three rungs -- (0.8, 1.0, 1.25) -- and the `binary` front-end
|
||||
sweeps none at all. Measured on stamped marks over controlled backgrounds
|
||||
(`scripts/detector_response.py`), the response is a COMB: doubao scores 0.99 exactly at
|
||||
each rung and collapses to 0.37-0.48 between them, against a 0.50 gate. So a mark whose
|
||||
@@ -93,7 +93,10 @@ def wilson(k: int, n: int, z: float = 1.96) -> tuple[float, float]:
|
||||
|
||||
|
||||
def score_at_scales(engine: Any, image: np.ndarray, scales: tuple[float, ...]) -> dict[float, float]:
|
||||
"""`_tophat_score`'s inner loop, opened up so the ladder is a parameter.
|
||||
"""`_ladder_best`'s inner loop, opened up so the ladder is a parameter.
|
||||
|
||||
A DELIBERATE second copy of the sweep: it exists to measure ladders the shipped
|
||||
engine does not have, so it must not be folded back into `_ladder_best`.
|
||||
|
||||
Deliberately reaches into the engine (`tophat_response`, `_glyph_silhouette`): a
|
||||
measurement script may, product code may not. Kept a faithful copy of the shipped
|
||||
|
||||
@@ -62,17 +62,10 @@ OUT = REPO / ".local-eval" / "vendor-cohorts.jsonl"
|
||||
FIRED = REPO / ".local-eval" / "visible-positives.jsonl"
|
||||
SHEET_DIR = REPO / ".local-eval" / "vendor-cohort-sheets"
|
||||
|
||||
# A producer code is `001` + `1` + USCC(18) + a 5-digit app/product suffix, so two
|
||||
# codes sharing the USCC are the same legal entity registering different products.
|
||||
# Slicing is defensive: anything not matching the layout is grouped by its raw value.
|
||||
_USCC_START, _USCC_END = 4, 22
|
||||
|
||||
|
||||
def uscc_of(code: str) -> str:
|
||||
"""The 18-char Unified Social Credit Code embedded in a TC260 producer code."""
|
||||
if len(code) >= _USCC_END and code[:3] == "001":
|
||||
return code[_USCC_START:_USCC_END]
|
||||
return code
|
||||
# `uscc_of` moved into the library (`metadata.uscc_of`) when the USCC -> vendor table
|
||||
# started driving `api.visible_provenance`; this script must group by the same rule the
|
||||
# product uses, so it imports rather than reimplements it.
|
||||
from remove_ai_watermarks.metadata import uscc_of # noqa: E402
|
||||
|
||||
|
||||
def _one(path_str: str) -> dict[str, Any] | None:
|
||||
|
||||
@@ -97,7 +97,7 @@ def _score(args: ScoreArgs) -> dict[str, Any] | None:
|
||||
eng = TextMarkEngine(build_config(asset, name, basis, overrides))
|
||||
loc = eng.locate(img)
|
||||
try:
|
||||
score, box = eng._tophat_best(img, loc)
|
||||
score, box = eng._ladder_best(img, loc)
|
||||
except Exception:
|
||||
return None
|
||||
return {"path": path_str, "score": round(float(score), 4), "box": box}
|
||||
@@ -225,7 +225,7 @@ def sheets(pos: list[dict[str, Any]], name: str, per_sheet: int = 24) -> None:
|
||||
# genuinely unknown, which is the one case a dense ladder earns its cost -- but it also
|
||||
# hands clean corners extra chances to match, so it must never set a gate.
|
||||
_FIT_SCALES = tuple(round(0.4 * (1.03**i), 4) for i in range(80)) # 0.40 .. ~4.1
|
||||
# The ladder the product actually ships (`_text_mark_engine._tophat_best`).
|
||||
# The ladder the product actually ships (`_text_mark_engine._ladder_best`).
|
||||
_SHIPPED_LADDER = (0.8, 1.0, 1.25)
|
||||
|
||||
|
||||
@@ -460,7 +460,7 @@ def _cross_score(args: tuple[str, Any, Any]) -> dict[str, Any] | None:
|
||||
eng = TextMarkEngine(cfg)
|
||||
loc = eng.locate(img)
|
||||
try:
|
||||
score, _ = eng._tophat_best(img, loc)
|
||||
score, _ = eng._ladder_best(img, loc)
|
||||
except Exception:
|
||||
return None
|
||||
out[key] = round(float(score), 4)
|
||||
|
||||
@@ -35,9 +35,15 @@ _warnings.filterwarnings("ignore", message=r".*ImageProcessorFast.*")
|
||||
__version__ = "0.25.0"
|
||||
|
||||
__all__ = [
|
||||
"BatchSummary",
|
||||
"InvisibleOptions",
|
||||
"MetadataStripIncomplete",
|
||||
"RemoveAllResult",
|
||||
"__version__",
|
||||
"identify_video",
|
||||
"inspect_video_metadata",
|
||||
"remove_all",
|
||||
"remove_batch",
|
||||
"remove_video_all",
|
||||
"remove_video_batch",
|
||||
"remove_video_invisible",
|
||||
@@ -48,7 +54,16 @@ __all__ = [
|
||||
]
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from remove_ai_watermarks.api import remove_visible, visible_provenance
|
||||
from remove_ai_watermarks.api import (
|
||||
BatchSummary,
|
||||
InvisibleOptions,
|
||||
MetadataStripIncomplete,
|
||||
RemoveAllResult,
|
||||
remove_all,
|
||||
remove_batch,
|
||||
remove_visible,
|
||||
visible_provenance,
|
||||
)
|
||||
from remove_ai_watermarks.video import (
|
||||
identify_video,
|
||||
inspect_video_metadata,
|
||||
@@ -63,7 +78,16 @@ if TYPE_CHECKING:
|
||||
def __getattr__(name: str) -> object:
|
||||
"""Lazily resolve the high-level API (PEP 562), so the heavy imports (cv2, the
|
||||
metadata/identify stack) load only when a caller actually reaches for them."""
|
||||
if name in ("remove_visible", "visible_provenance"):
|
||||
if name in (
|
||||
"BatchSummary",
|
||||
"InvisibleOptions",
|
||||
"MetadataStripIncomplete",
|
||||
"RemoveAllResult",
|
||||
"remove_all",
|
||||
"remove_batch",
|
||||
"remove_visible",
|
||||
"visible_provenance",
|
||||
):
|
||||
from remove_ai_watermarks import api
|
||||
|
||||
return getattr(api, name)
|
||||
|
||||
@@ -329,14 +329,44 @@ def _extract_c2pa_info_png(image_path: Path) -> dict[str, Any]:
|
||||
return info
|
||||
|
||||
|
||||
def extract_c2pa_info(image_path: Path) -> dict[str, Any]:
|
||||
"""Return normalized C2PA evidence from the official reader or PNG fallback."""
|
||||
def _extract_c2pa_info_impl(image_path: Path) -> dict[str, Any]:
|
||||
store = read_manifest_store_json(Path(image_path))
|
||||
if store is not None:
|
||||
return c2pa_info_from_manifest_store(store)
|
||||
return _extract_c2pa_info_png(Path(image_path))
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=4)
|
||||
def _extract_c2pa_info_cached(path_str: str, _mtime_ns: int, _size: int, _reader: bool) -> dict[str, Any]:
|
||||
"""Cache shim: every argument after the path is key-only.
|
||||
|
||||
``_reader`` is in the key because the answer genuinely depends on it -- with the
|
||||
official reader the manifest comes back as a store, without it from the hand-rolled
|
||||
PNG chunk parser, and the two produce different ``c2pa_manifest`` labels. Keying on
|
||||
the file alone handed a reader-path result to a caller that had disabled the reader.
|
||||
"""
|
||||
return _extract_c2pa_info_impl(Path(path_str))
|
||||
|
||||
|
||||
def extract_c2pa_info(image_path: Path) -> dict[str, Any]:
|
||||
"""Return normalized C2PA evidence from the official reader or PNG fallback.
|
||||
|
||||
Memoized on ``(path, mtime_ns, size)``: one ``identify`` reaches this twice (once
|
||||
directly, once inside ``get_ai_metadata``) and each call re-runs the Rust manifest
|
||||
reader and re-parses its JSON. Size joins mtime in the key because this package
|
||||
rewrites files in place, and an in-place rewrite can land inside one mtime tick.
|
||||
"""
|
||||
try:
|
||||
stat = image_path.stat()
|
||||
except OSError:
|
||||
# No stat (a pipe, or a race): read uncached rather than fail.
|
||||
return _extract_c2pa_info_impl(image_path)
|
||||
cached = _extract_c2pa_info_cached(str(image_path), stat.st_mtime_ns, stat.st_size, _C2PA_READER_AVAILABLE)
|
||||
# Deep-ish copy: values are scalars plus a few lists, and a caller mutating one of
|
||||
# those lists would otherwise poison every later reader of the same file.
|
||||
return {key: list(cast("list[Any]", value)) if isinstance(value, list) else value for key, value in cached.items()}
|
||||
|
||||
|
||||
def inject_c2pa_chunk(target_path: Path, output_path: Path, c2pa_chunk: bytes) -> None:
|
||||
"""Replace any C2PA chunks in a PNG and insert ``c2pa_chunk`` before IDAT."""
|
||||
if target_path.suffix.casefold() != ".png" or output_path.suffix.casefold() != ".png":
|
||||
|
||||
@@ -147,3 +147,15 @@ AI_GENERATOR_TOKENS = frozenset(
|
||||
|
||||
_C2PA_ACTION_NAMES = _tokens("created|converted|edited|filtered|cropped|resized|opened|placed")
|
||||
C2PA_ACTIONS = {f"c2pa.{action}".encode(): action for action in _C2PA_ACTION_NAMES}
|
||||
|
||||
|
||||
# TC260 producer identity -> the mark key whose vendor signs with it now lives on the
|
||||
# registry rows (``KnownMark.tc260_producer_codes``, read through
|
||||
# ``watermark_registry.tc260_producer_vendors``). Keeping the codes beside the mark is
|
||||
# what stops a newly registered TC260 vendor from silently falling back to ByteDance.
|
||||
#
|
||||
# What a TC260 label confirms when its producer is absent or unmapped. Historical
|
||||
# behaviour, kept as the fallback so an unrecognized producer never regresses to no
|
||||
# relaxation at all: ByteDance's two products are the ones the relaxed band was
|
||||
# calibrated on (see _text_mark_engine._DEFAULT_PROVENANCE_NCC_FACTOR).
|
||||
TC260_FALLBACK_VENDORS: frozenset[str] = frozenset({"doubao", "jimeng"})
|
||||
|
||||
@@ -561,11 +561,18 @@ def _clip_sam_masks_to_boxes(
|
||||
image_size: tuple[int, int],
|
||||
) -> list[np.ndarray]:
|
||||
"""Match Impact Pack by intersecting each SAM mask with its detector box."""
|
||||
# Same rectangle primitive the shared fill uses, rather than a private zeros/fill
|
||||
# copy. `dilate=0` because this box CLIPS a SAM mask -- growing it would admit the
|
||||
# pixels the clip exists to exclude. The function-local import keeps region_eraser's
|
||||
# module-scope cv2 off this module's import path.
|
||||
from remove_ai_watermarks.region_eraser import boxes_to_mask
|
||||
|
||||
width, height = image_size
|
||||
clipped: list[np.ndarray] = []
|
||||
for mask, (x1, y1, x2, y2) in zip(masks, boxes, strict=True):
|
||||
box_mask = np.zeros((height, width), dtype=np.uint8)
|
||||
box_mask[max(0, y1) : min(height, y2), max(0, x1) : min(width, x2)] = 255
|
||||
# (x, y, w, h) with w = x2 - x1 keeps x + w == x2, so a negative origin clamps
|
||||
# to the same span the explicit max/min pair produced.
|
||||
box_mask = boxes_to_mask((height, width), [(x1, y1, x2 - x1, y2 - y1)], dilate=0)
|
||||
clipped.append(np.bitwise_and(mask.astype(np.uint8), box_mask))
|
||||
return clipped
|
||||
|
||||
|
||||
@@ -10,7 +10,6 @@ registry and every identify path still run anywhere.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import TYPE_CHECKING, Literal
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -71,19 +70,12 @@ SDXL_ZIMAGE_GEMINI_STRENGTH = 0.25
|
||||
SDXL_ZIMAGE_UNKNOWN_STRENGTH = SDXL_ZIMAGE_GEMINI_STRENGTH
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _StrengthPolicy:
|
||||
unknown: float
|
||||
by_vendor: dict[str, float]
|
||||
|
||||
def choose(self, vendor: str | None) -> float:
|
||||
return self.by_vendor.get((vendor or "").casefold(), self.unknown)
|
||||
|
||||
|
||||
_SDXL_ZIMAGE_POLICY = _StrengthPolicy(
|
||||
unknown=SDXL_ZIMAGE_UNKNOWN_STRENGTH,
|
||||
by_vendor={"openai": SDXL_ZIMAGE_OPENAI_STRENGTH, "google": SDXL_ZIMAGE_GEMINI_STRENGTH},
|
||||
)
|
||||
# sdxl-zimage picks its strength from the VENDOR (unlike qwen-zimage, which derives it
|
||||
# from image area). An unlisted or unknown vendor falls back to the Gemini value.
|
||||
_SDXL_ZIMAGE_STRENGTH_BY_VENDOR: dict[str, float] = {
|
||||
"openai": SDXL_ZIMAGE_OPENAI_STRENGTH,
|
||||
"google": SDXL_ZIMAGE_GEMINI_STRENGTH,
|
||||
}
|
||||
_ALIASES = {
|
||||
"qwen_zimage": QWEN_ZIMAGE_PROFILE,
|
||||
"sdxl_zimage": SDXL_ZIMAGE_PROFILE,
|
||||
@@ -134,7 +126,7 @@ def resolve_strength(
|
||||
if strength is not None:
|
||||
return strength
|
||||
if normalize_profile(pipeline or "") == SDXL_ZIMAGE_PROFILE:
|
||||
return _SDXL_ZIMAGE_POLICY.choose(vendor)
|
||||
return _SDXL_ZIMAGE_STRENGTH_BY_VENDOR.get((vendor or "").casefold(), SDXL_ZIMAGE_UNKNOWN_STRENGTH)
|
||||
if size is None:
|
||||
raise ValueError("qwen-zimage resolves strength from image area, so size is required")
|
||||
from remove_ai_watermarks._internal.qwen_zimage_pipeline import resolution_adaptive_denoise
|
||||
|
||||
@@ -132,7 +132,7 @@ class TextMarkConfig:
|
||||
# Which image dimension the mark's size and margins scale with. VENDOR-SPECIFIC,
|
||||
# measured, not assumed -- see TextMarkEngine.scale_base. "short" = min(h, w), "width" = w.
|
||||
scale_basis: Literal["short", "width"] = "width"
|
||||
# Scale rungs ``_tophat_best`` sweeps (the detection comb). PER-MARK: a vendor
|
||||
# Scale rungs ``_ladder_best`` sweeps (the detection comb). PER-MARK: a vendor
|
||||
# whose stamp sizes do not land on the shared 3-rung comb carries its own ladder
|
||||
# (measured for 千问, whose marks sit in two size modes ~1.6x apart -- one fraction
|
||||
# on 3 rungs covers only ~75% of them). Densifying the SHARED ladder for everyone
|
||||
@@ -170,6 +170,33 @@ class TextMarkDetection:
|
||||
confidence: float = 0.0
|
||||
region: tuple[int, int, int, int] = (0, 0, 0, 0)
|
||||
coverage: float = 0.0 # fraction of the box occupied by glyph pixels
|
||||
# ROI-local (x0, y0, x1, y1) of the ladder sweep's best match, in the LOCATED BOX's
|
||||
# coordinates. None for the ``binary`` front-end (it runs no sweep) and whenever no
|
||||
# rung matched. ``footprint_mask`` bounds the fill with it, so carrying it here is
|
||||
# what stops the mask path re-running a sweep the detector already ran.
|
||||
match_box: tuple[int, int, int, int] | None = None
|
||||
# The trust level this detection was taken at, mirroring detect()'s ``provenance``.
|
||||
# ``footprint_mask`` reuses a threaded detection only when it matches the STRICT
|
||||
# level its own re-detect would have used -- see TextMarkEngine._strict_detection.
|
||||
provenance: bool = False
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TextMarkScan:
|
||||
"""The trust-level-BLIND half of text-mark detection, reusable across both levels.
|
||||
|
||||
``loc is None`` means detection stopped before any scan (empty or too-small image).
|
||||
``score is None`` means the binary front-end fell below its coverage gate, which is
|
||||
a verdict of "not detected, confidence 0.0" without consulting the rival margin.
|
||||
"""
|
||||
|
||||
loc: TextMarkLocation | None
|
||||
box: NDArray[Any] | None # box-sized binary glyph mask
|
||||
base: int # scale_base(image)
|
||||
frame: tuple[int, int] = (0, 0) # (h, w) of the scanned image
|
||||
coverage: float = 0.0
|
||||
score: float | None = None
|
||||
match_box: tuple[int, int, int, int] | None = None
|
||||
|
||||
|
||||
# Alpha / silhouette templates, cached per asset name. This shared cache lets every
|
||||
@@ -306,6 +333,28 @@ class TextMarkEngine:
|
||||
|
||||
# ── Locate ──────────────────────────────────────────────────────────
|
||||
|
||||
def _roi_fields(
|
||||
self, image: NDArray[Any], loc: TextMarkLocation
|
||||
) -> tuple[NDArray[Any], NDArray[Any], NDArray[Any]]:
|
||||
"""``(luma, saturation, local_background)`` for the located box, all float32.
|
||||
|
||||
The ROI is normalized to 3-channel BGR first (grayscale / BGRA would break
|
||||
``axis=2``).
|
||||
|
||||
Local background model: a strong Gaussian blur (sigma ~ box height); the white
|
||||
top-hat (``luma - local_bg``) lights up bright thin strokes regardless of the
|
||||
absolute background level.
|
||||
|
||||
The callers each keep their own ``bh < 16 or bw < 16`` guard: they return three
|
||||
different sentinels for a degenerate ROI, so the check cannot move in here.
|
||||
"""
|
||||
x, y, bw, bh = loc.bbox
|
||||
roi = image_io.to_bgr(image[y : y + bh, x : x + bw]).astype(np.float32)
|
||||
luma = roi.mean(axis=2)
|
||||
sat = roi.max(axis=2) - roi.min(axis=2)
|
||||
sigma = max(4.0, bh * 0.4) # 0.4 factor and 4.0 floor are calibrated; do not retune
|
||||
return luma, sat, cv2.GaussianBlur(luma, (0, 0), sigmaX=sigma, sigmaY=sigma)
|
||||
|
||||
def tophat_response(self, image: NDArray[Any], loc: TextMarkLocation) -> NDArray[Any] | None:
|
||||
"""The CONTINUOUS white top-hat in the located box -- the glyph signal, unbinarized.
|
||||
|
||||
@@ -328,26 +377,48 @@ class TextMarkEngine:
|
||||
Kept per-mark (``detect_frontend``) rather than switched globally, because a
|
||||
front-end change must be measured per mark before it ships.
|
||||
"""
|
||||
return self._residual_response(image, loc, absolute=False)
|
||||
|
||||
def _residual_response(self, image: NDArray[Any], loc: TextMarkLocation, *, absolute: bool) -> NDArray[Any] | None:
|
||||
"""Max-normalized uint8 local-luma residual in the located box, saturation-weighted.
|
||||
|
||||
``absolute=False`` keeps only the POSITIVE side -- the white top-hat, for a mark
|
||||
always rendered brighter than its background. ``absolute=True`` takes the
|
||||
magnitude, for a renderer that switches between light-on-dark and dark-on-light
|
||||
while preserving one silhouette; a one-polarity top-hat misses the latter.
|
||||
"""
|
||||
c = self.config
|
||||
x, y, bw, bh = loc.bbox
|
||||
_x, _y, bw, bh = loc.bbox
|
||||
if bh < 16 or bw < 16:
|
||||
return None
|
||||
roi = image_io.to_bgr(image[y : y + bh, x : x + bw]).astype(np.float32)
|
||||
luma = roi.mean(axis=2)
|
||||
sat = roi.max(axis=2) - roi.min(axis=2)
|
||||
sigma = max(4.0, bh * 0.4)
|
||||
tophat = luma - cv2.GaussianBlur(luma, (0, 0), sigmaX=sigma, sigmaY=sigma)
|
||||
resp = np.clip(tophat, 0, None) * (sat < c.max_saturation)
|
||||
luma, sat, local_bg = self._roi_fields(image, loc)
|
||||
residual = luma - local_bg
|
||||
resp = (np.abs(residual) if absolute else np.clip(residual, 0, None)) * (sat < c.max_saturation)
|
||||
peak = float(resp.max())
|
||||
if peak <= 1e-6:
|
||||
return None
|
||||
return (resp / peak * 255).astype(np.uint8)
|
||||
|
||||
def _tophat_best(
|
||||
def _detect_response(self, image: NDArray[Any], loc: TextMarkLocation) -> NDArray[Any] | None:
|
||||
"""The uint8 image the ladder sweep correlates the silhouette against, chosen by
|
||||
``TextMarkConfig.detect_frontend``. ``binary`` runs no sweep and never reaches here."""
|
||||
frontend = self.config.detect_frontend
|
||||
if frontend == "gray":
|
||||
x, y, bw, bh = loc.bbox
|
||||
if bh < 16 or bw < 16:
|
||||
return None
|
||||
return cv2.cvtColor(image_io.to_bgr(image[y : y + bh, x : x + bw]), cv2.COLOR_BGR2GRAY)
|
||||
if frontend == "contrast":
|
||||
return self._residual_response(image, loc, absolute=True)
|
||||
if frontend == "tophat":
|
||||
return self._residual_response(image, loc, absolute=False)
|
||||
raise ValueError(f"{frontend!r} has no ladder response (binary runs no sweep)")
|
||||
|
||||
def _ladder_best(
|
||||
self, image: NDArray[Any], loc: TextMarkLocation
|
||||
) -> tuple[float, tuple[int, int, int, int] | None]:
|
||||
"""Best TM_CCOEFF_NORMED of a soft template against the continuous response, and
|
||||
the ROI-local box (x0, y0, x1, y1) where that best match sits.
|
||||
"""Best TM_CCOEFF_NORMED of the mark's silhouette against its front-end response,
|
||||
and the ROI-local box (x0, y0, x1, y1) where that best match sits.
|
||||
|
||||
Sweeps the mark's scale ladder: the nominal glyph size is derived from the mark's
|
||||
geometry, but a vendor re-rasterization shifts it by a few percent and the
|
||||
@@ -358,10 +429,15 @@ class TextMarkEngine:
|
||||
detection, the box bounds the fill. Sharing it is deliberate: the standing rule is
|
||||
that detection and the mask use the same front-end, and the way that rule was last
|
||||
broken was a drift between two separate implementations. One method makes the drift
|
||||
impossible instead of merely discouraged.
|
||||
impossible instead of merely discouraged -- which is why the three continuous
|
||||
front-ends (tophat / contrast / gray) sweep here rather than in a copy each.
|
||||
|
||||
Tie-breaking is load-bearing: the ``0.0`` seed plus the STRICT ``>`` means the
|
||||
EARLIEST ladder rung wins a tie, and a sweep whose maximum is not above 0.0
|
||||
returns no box at all.
|
||||
"""
|
||||
c = self.config
|
||||
resp = self.tophat_response(image, loc)
|
||||
resp = self._detect_response(image, loc)
|
||||
sil = self._glyph_silhouette()
|
||||
if resp is None or sil is None:
|
||||
return (0.0, None)
|
||||
@@ -373,90 +449,12 @@ class TextMarkEngine:
|
||||
gh = max(4, int(c.alpha_height_frac * base * scale))
|
||||
if gw >= resp.shape[1] or gh >= resp.shape[0]:
|
||||
continue
|
||||
tmpl = cv2.resize(sil, (gw, gh), interpolation=cv2.INTER_AREA).astype(np.float32)
|
||||
if c.template_blur > 0:
|
||||
tmpl = cv2.GaussianBlur(tmpl, (0, 0), sigmaX=c.template_blur, sigmaY=c.template_blur)
|
||||
result = cv2.matchTemplate(resp, tmpl.astype(np.uint8), cv2.TM_CCOEFF_NORMED)
|
||||
_, score, _, top_left = cv2.minMaxLoc(result)
|
||||
if score > best_score:
|
||||
tx, ty = int(top_left[0]), int(top_left[1])
|
||||
best_score, best_box = float(score), (tx, ty, tx + gw - 1, ty + gh - 1)
|
||||
return (best_score, best_box)
|
||||
|
||||
def _tophat_score(self, image: NDArray[Any], loc: TextMarkLocation) -> float:
|
||||
"""The detection score alone -- the box the removal mask needs is discarded here."""
|
||||
return self._tophat_best(image, loc)[0]
|
||||
|
||||
def _contrast_best(
|
||||
self, image: NDArray[Any], loc: TextMarkLocation
|
||||
) -> tuple[float, tuple[int, int, int, int] | None]:
|
||||
"""Best silhouette match against the absolute local-luma residual.
|
||||
|
||||
Unlike the white top-hat, this response is polarity-independent: the same
|
||||
watermark can be lighter or darker than its local background. Detection and
|
||||
removal share the returned box, preserving the front-end parity contract.
|
||||
"""
|
||||
c = self.config
|
||||
x, y, bw, bh = loc.bbox
|
||||
if bh < 16 or bw < 16:
|
||||
return (0.0, None)
|
||||
roi = image_io.to_bgr(image[y : y + bh, x : x + bw]).astype(np.float32)
|
||||
luma = roi.mean(axis=2)
|
||||
sat = roi.max(axis=2) - roi.min(axis=2)
|
||||
sigma = max(4.0, bh * 0.4)
|
||||
response = np.abs(luma - cv2.GaussianBlur(luma, (0, 0), sigmaX=sigma, sigmaY=sigma))
|
||||
response *= sat < c.max_saturation
|
||||
peak = float(response.max())
|
||||
sil = self._glyph_silhouette()
|
||||
if peak <= 1e-6 or sil is None:
|
||||
return (0.0, None)
|
||||
response = (response / peak * 255).astype(np.uint8)
|
||||
base = self.scale_base(image)
|
||||
best_score = 0.0
|
||||
best_box: tuple[int, int, int, int] | None = None
|
||||
for scale in c.ladder:
|
||||
gw = max(c.min_gw, int(c.alpha_width_frac * base * scale))
|
||||
gh = max(4, int(c.alpha_height_frac * base * scale))
|
||||
if gw >= response.shape[1] or gh >= response.shape[0]:
|
||||
continue
|
||||
template = cv2.resize(sil, (gw, gh), interpolation=cv2.INTER_AREA)
|
||||
result = cv2.matchTemplate(response, template, cv2.TM_CCOEFF_NORMED)
|
||||
_, score, _, top_left = cv2.minMaxLoc(result)
|
||||
if score > best_score:
|
||||
tx, ty = int(top_left[0]), int(top_left[1])
|
||||
best_score, best_box = float(score), (tx, ty, tx + gw - 1, ty + gh - 1)
|
||||
return (best_score, best_box)
|
||||
|
||||
def _gray_best(self, image: NDArray[Any], loc: TextMarkLocation) -> tuple[float, tuple[int, int, int, int] | None]:
|
||||
"""Best TM_CCOEFF_NORMED of the silhouette against the raw GRAYSCALE ROI, and
|
||||
the ROI-local box (x0, y0, x1, y1) of that best match.
|
||||
|
||||
Mirrors :meth:`_tophat_best` (same ladder sweep, same one-method contract so
|
||||
detection and the removal mask can never drift), but skips the top-hat
|
||||
entirely: the RunningHub mark is a faint mid-gray text the top-hat's
|
||||
background subtraction suppresses to clean-arm levels, while raw gray NCC
|
||||
separates (see ``TextMarkConfig.detect_frontend``). Contrast-DEPENDENT by
|
||||
construction, so the gate must be picked against the clean arm, which is
|
||||
what ``scripts/vendor_mark_calibrate.py`` does.
|
||||
"""
|
||||
c = self.config
|
||||
x, y, bw, bh = loc.bbox
|
||||
if bh < 16 or bw < 16:
|
||||
return (0.0, None)
|
||||
roi = cv2.cvtColor(image_io.to_bgr(image[y : y + bh, x : x + bw]), cv2.COLOR_BGR2GRAY)
|
||||
sil = self._glyph_silhouette()
|
||||
if sil is None:
|
||||
return (0.0, None)
|
||||
base = self.scale_base(image)
|
||||
best_score = 0.0
|
||||
best_box: tuple[int, int, int, int] | None = None
|
||||
for scale in c.ladder:
|
||||
gw = max(c.min_gw, int(c.alpha_width_frac * base * scale))
|
||||
gh = max(4, int(c.alpha_height_frac * base * scale))
|
||||
if gw >= roi.shape[1] or gh >= roi.shape[0]:
|
||||
continue
|
||||
tmpl = cv2.resize(sil, (gw, gh), interpolation=cv2.INTER_AREA)
|
||||
result = cv2.matchTemplate(roi, tmpl, cv2.TM_CCOEFF_NORMED)
|
||||
if c.detect_frontend == "tophat" and c.template_blur > 0:
|
||||
tmpl = cv2.GaussianBlur(
|
||||
tmpl.astype(np.float32), (0, 0), sigmaX=c.template_blur, sigmaY=c.template_blur
|
||||
).astype(np.uint8)
|
||||
result = cv2.matchTemplate(resp, tmpl, cv2.TM_CCOEFF_NORMED)
|
||||
_, score, _, top_left = cv2.minMaxLoc(result)
|
||||
if score > best_score:
|
||||
tx, ty = int(top_left[0]), int(top_left[1])
|
||||
@@ -570,10 +568,33 @@ class TextMarkEngine:
|
||||
the mark's own ``provenance_ncc_factor`` to recover a faint or slightly
|
||||
re-rendered mark (per-mark, not shared -- see _DEFAULT_PROVENANCE_NCC_FACTOR).
|
||||
"""
|
||||
scan = self._scan(image)
|
||||
return self._verdict(scan, provenance=provenance)
|
||||
|
||||
def detect_both(self, image: NDArray[Any] | None) -> tuple[TextMarkDetection, TextMarkDetection]:
|
||||
"""``(strict, relaxed)`` from ONE scan of the image.
|
||||
|
||||
``provenance`` scales the acceptance THRESHOLD and nothing else -- the locate
|
||||
box, the glyph mask, the coverage and the front-end ladder score are computed
|
||||
identically at either trust level. Two ``detect`` calls therefore ran the same
|
||||
expensive sweep twice to reach two verdicts, which is what the arbiter's
|
||||
perception pass did for every mark on every image.
|
||||
|
||||
Returns two DISTINCT objects: subclasses demote a verdict by mutating it.
|
||||
"""
|
||||
scan = self._scan(image)
|
||||
return self._verdict(scan, provenance=False), self._verdict(scan, provenance=True)
|
||||
|
||||
def _scan(self, image: NDArray[Any] | None) -> TextMarkScan:
|
||||
"""Everything in detection that does not depend on the trust level.
|
||||
|
||||
Per-CALL only, never memoized on ``self``: ``remove_auto_marks`` re-invokes each
|
||||
engine on a progressively cleaned frame inside one process, so a cached scan
|
||||
would answer for the wrong pixels.
|
||||
"""
|
||||
c = self.config
|
||||
det = TextMarkDetection()
|
||||
if image is None or image.size == 0:
|
||||
return det
|
||||
return TextMarkScan(None, None, 0)
|
||||
# Guard against the small-image NCC-noise false positive (see
|
||||
# _MIN_DETECT_SHORT_SIDE): an icon/thumbnail is too small to carry a real
|
||||
# text label, and the degraded few-pixel template spuriously correlates.
|
||||
@@ -584,57 +605,57 @@ class TextMarkEngine:
|
||||
min(image.shape[:2]),
|
||||
_MIN_DETECT_SHORT_SIDE,
|
||||
)
|
||||
return det
|
||||
return TextMarkScan(None, None, 0)
|
||||
loc = self.locate(image)
|
||||
box = self.extract_mask(image, loc) # box-sized mask (== old full-frame cropped to bbox)
|
||||
_x, _y, bw, bh = loc.bbox
|
||||
coverage = float((box > 0).sum()) / float(max(1, bw * bh))
|
||||
det.region = loc.bbox
|
||||
det.coverage = coverage
|
||||
if c.detect_frontend == "tophat":
|
||||
# The continuous front-end does not depend on the binarized blob, so the
|
||||
# coverage gate (a blob-area heuristic) does not apply to it.
|
||||
score = self._tophat_score(image, loc)
|
||||
threshold = c.detect_ncc_threshold * (c.provenance_ncc_factor if provenance else 1.0)
|
||||
det.confidence = score
|
||||
det.detected = score >= threshold and self._rival_margin_ok(score, box, self.scale_base(image))
|
||||
logger.debug("%s detect (tophat): ncc=%.2f thr=%.2f detected=%s", c.name, score, threshold, det.detected)
|
||||
base = self.scale_base(image)
|
||||
match_box: tuple[int, int, int, int] | None = None
|
||||
if c.detect_frontend == "binary":
|
||||
# The coverage gate is a blob-AREA heuristic, so it applies only to the
|
||||
# front-end that binarizes; the continuous ones never build a blob. Below
|
||||
# the gate the detection stays at confidence 0.0 and the rival margin is
|
||||
# never consulted, so the score stays None here.
|
||||
score = self._template_match_score(box, base) if coverage >= c.detect_min_coverage else None
|
||||
else:
|
||||
score, match_box = self._ladder_best(image, loc)
|
||||
return TextMarkScan(loc, box, base, frame=image.shape[:2], coverage=coverage, score=score, match_box=match_box)
|
||||
|
||||
def _verdict(self, scan: TextMarkScan, *, provenance: bool) -> TextMarkDetection:
|
||||
"""Apply the trust-level-dependent tail to a scan, as a fresh result object."""
|
||||
c = self.config
|
||||
det = TextMarkDetection(provenance=provenance)
|
||||
if scan.loc is None or scan.box is None:
|
||||
return det
|
||||
if c.detect_frontend == "gray":
|
||||
# Same no-coverage-gate reasoning as tophat: the gray front-end never
|
||||
# binarizes, so a blob-area heuristic does not apply to it either.
|
||||
score = self._gray_best(image, loc)[0]
|
||||
threshold = c.detect_ncc_threshold * (c.provenance_ncc_factor if provenance else 1.0)
|
||||
det.confidence = score
|
||||
det.detected = score >= threshold and self._rival_margin_ok(score, box, self.scale_base(image))
|
||||
logger.debug("%s detect (gray): ncc=%.2f thr=%.2f detected=%s", c.name, score, threshold, det.detected)
|
||||
det.region = scan.loc.bbox
|
||||
det.coverage = scan.coverage
|
||||
det.match_box = scan.match_box
|
||||
if scan.score is None: # binary front-end below the coverage gate
|
||||
return det
|
||||
if c.detect_frontend == "contrast":
|
||||
score = self._contrast_best(image, loc)[0]
|
||||
threshold = c.detect_ncc_threshold * (c.provenance_ncc_factor if provenance else 1.0)
|
||||
det.confidence = score
|
||||
det.detected = score >= threshold and self._rival_margin_ok(score, box, self.scale_base(image))
|
||||
logger.debug(
|
||||
"%s detect (contrast): ncc=%.2f thr=%.2f detected=%s",
|
||||
c.name,
|
||||
score,
|
||||
threshold,
|
||||
det.detected,
|
||||
)
|
||||
return det
|
||||
if coverage >= c.detect_min_coverage:
|
||||
score = self._template_match_score(box, self.scale_base(image))
|
||||
threshold = c.detect_ncc_threshold * (c.provenance_ncc_factor if provenance else 1.0)
|
||||
det.confidence = score
|
||||
det.detected = score >= threshold and self._rival_margin_ok(score, box, self.scale_base(image))
|
||||
logger.debug(
|
||||
"%s detect: coverage=%.3f ncc=%.2f thr=%.2f detected=%s",
|
||||
c.name,
|
||||
coverage,
|
||||
score,
|
||||
threshold,
|
||||
det.detected,
|
||||
)
|
||||
threshold = c.detect_ncc_threshold * (c.provenance_ncc_factor if provenance else 1.0)
|
||||
det.confidence = scan.score
|
||||
# Short-circuit is load-bearing: _rival_margin_ok scores every rival template
|
||||
# and logs its own rejection line, so it must stay unevaluated below threshold.
|
||||
det.detected = scan.score >= threshold and self._rival_margin_ok(scan.score, scan.box, scan.base)
|
||||
logger.debug(
|
||||
"%s detect (%s): coverage=%.3f ncc=%.2f thr=%.2f detected=%s",
|
||||
c.name,
|
||||
c.detect_frontend,
|
||||
scan.coverage,
|
||||
scan.score,
|
||||
threshold,
|
||||
det.detected,
|
||||
)
|
||||
return self._post_gate(det, scan)
|
||||
|
||||
def _post_gate(self, det: TextMarkDetection, scan: TextMarkScan) -> TextMarkDetection:
|
||||
"""Per-mark demotion applied after the shared threshold, for both trust levels.
|
||||
|
||||
OVERRIDABLE. It lives here rather than in a ``detect`` override so a mark's gate
|
||||
cannot be silently skipped by the single-pass ``detect_both`` path -- which is
|
||||
exactly what happened while the anchor demotions were ``detect`` overrides.
|
||||
"""
|
||||
return det
|
||||
|
||||
# ── Inpaint footprint (for the inpaint-fallback removal path) ────────
|
||||
@@ -644,8 +665,132 @@ class TextMarkEngine:
|
||||
# to mask. A real strip covers hundreds of pixels.
|
||||
_MIN_GLYPH_PIXELS = 20
|
||||
|
||||
def _strict_detection(self, image: NDArray[Any], detection: TextMarkDetection | None) -> TextMarkDetection:
|
||||
"""The STRICT detection the footprint is bounded by.
|
||||
|
||||
A threaded detection is reused only when it was taken at the same strict level
|
||||
this method would have used itself. A provenance-RELAXED detection is NOT
|
||||
reused: a strict re-detect can demote a mark the relaxed gate accepted, which
|
||||
for a continuous front-end means no mask at all. That is a MEASURED difference,
|
||||
not a refactor, so the strict semantics stay.
|
||||
|
||||
Reuse is safe against ``remove_auto_marks`` chaining marks on a progressively
|
||||
cleaned frame: the registry re-detects on that same cleaned array before
|
||||
threading (``KnownMark.localize``), so a threaded detection is never stale.
|
||||
"""
|
||||
if detection is not None and not detection.provenance:
|
||||
return detection
|
||||
return self.detect(image) # polymorphic: a subclass gate must still apply
|
||||
|
||||
def _geometry_rect(self, loc: TextMarkLocation, frame: tuple[int, int]) -> tuple[int, int, int, int]:
|
||||
"""The whole locate box, clamped to the frame -- the ``force`` footprint."""
|
||||
bx, by, bw, bh = loc.bbox
|
||||
h, w = frame
|
||||
return (bx, by, min(w, bx + bw), min(h, by + bh))
|
||||
|
||||
def _extend_match_box(
|
||||
self, box: tuple[int, int, int, int], loc: TextMarkLocation, frame: tuple[int, int]
|
||||
) -> tuple[int, int, int, int]:
|
||||
"""Grow an ROI-local box into the absolute fill rectangle by a symmetric pad.
|
||||
|
||||
OVERRIDABLE, and the override contract is specifically the DETECTOR'S MATCH BOX:
|
||||
a mark whose removable footprint reaches beyond what the NCC localizes -- Baidu's
|
||||
flat white tag right of the text run, LibLibAI's triangle logo left of the
|
||||
wordmark -- supplies its own extension here and inherits the rest of the
|
||||
footprint path. The blob-bbox branch never routes through an override.
|
||||
"""
|
||||
gx0, gy0, gx1, gy1 = box
|
||||
bx, by, _bw, bh = loc.bbox
|
||||
h, w = frame
|
||||
pad = max(4, int(0.10 * bh))
|
||||
return (
|
||||
max(0, bx + gx0 - pad),
|
||||
max(0, by + gy0 - pad),
|
||||
min(w, bx + gx1 + 1 + pad),
|
||||
min(h, by + gy1 + 1 + pad),
|
||||
)
|
||||
|
||||
def _footprint_rect(
|
||||
self,
|
||||
image: NDArray[Any],
|
||||
loc: TextMarkLocation,
|
||||
*,
|
||||
force: bool,
|
||||
detection: TextMarkDetection | None,
|
||||
) -> tuple[int, int, int, int] | None:
|
||||
"""Default footprint policy: the binary glyph blob's bbox, else the detector's
|
||||
own match box for the front-ends that under-segment, else the geometry box
|
||||
under ``force``.
|
||||
|
||||
The gray front-end exists for marks the top-hat under-segments, so the binary
|
||||
blob is NOT authoritative there: trusting it first bounded the fill by a PARTIAL
|
||||
blob (the faint head glyphs dropped out) and left the leftmost "Runni" of
|
||||
"RunningHub AI生成" unremoved (2026-07-22).
|
||||
|
||||
A dark-on-light Yuanbao mark has no WHITE top-hat blob at all, so the contrast
|
||||
front-end is bounded by the polarity-independent detector's match box too.
|
||||
|
||||
A tophat mark found only by the CONTINUOUS front-end has no binary glyph blob to
|
||||
bound, so the mask came back empty and removal was a silent no-op while
|
||||
``identify`` still reported the mark. Use the DETECTOR'S OWN best-match box: the
|
||||
correlation already located the mark at a position and scale, and thresholding
|
||||
the response was a strictly worse proxy for that. An earlier fix thresholded the
|
||||
max-normalized uint8 response at 0.5 -- which selects every non-zero pixel, not
|
||||
"half the peak" as its comment claimed -- and filled ~120% of the corner box on
|
||||
textured frames (measured: whole corner vs 58.7% for the match box, both
|
||||
detector-clean). Gated on an actual detection: on a clean corner the box would
|
||||
be spurious.
|
||||
"""
|
||||
ys, xs = np.where(self.extract_mask(image, loc) > 0)
|
||||
blob = (
|
||||
(int(xs.min()), int(ys.min()), int(xs.max()), int(ys.max())) if xs.size >= self._MIN_GLYPH_PIXELS else None
|
||||
)
|
||||
frontend = self.config.detect_frontend
|
||||
if frontend in ("gray", "contrast"):
|
||||
det = self._strict_detection(image, detection)
|
||||
box = det.match_box if det.detected else blob
|
||||
elif blob is not None:
|
||||
box = blob
|
||||
elif frontend == "tophat":
|
||||
det = self._strict_detection(image, detection)
|
||||
box = det.match_box if det.detected else None
|
||||
else:
|
||||
box = None
|
||||
if box is not None:
|
||||
return self._extend_match_box(box, loc, image.shape[:2])
|
||||
return self._geometry_rect(loc, image.shape[:2]) if force else None
|
||||
|
||||
def _match_box_rect(
|
||||
self,
|
||||
image: NDArray[Any],
|
||||
loc: TextMarkLocation,
|
||||
*,
|
||||
force: bool,
|
||||
detection: TextMarkDetection | None,
|
||||
) -> tuple[int, int, int, int] | None:
|
||||
"""Footprint policy for a mark whose fill must be bounded by the DETECTOR's match
|
||||
box and never by the binary glyph blob.
|
||||
|
||||
Baidu's white tag has a flat interior a top-hat cannot answer, and LibLibAI's
|
||||
blob bleeds up into background structure; in both cases the blob bbox is
|
||||
measurably wrong and the NCC match box is right. ``force`` takes priority here,
|
||||
unlike the default policy: a ``--no-detect`` caller named the mark, so the whole
|
||||
geometry box is the honest footprint.
|
||||
"""
|
||||
if force:
|
||||
return self._geometry_rect(loc, image.shape[:2])
|
||||
det = self._strict_detection(image, detection)
|
||||
if not det.detected or det.match_box is None:
|
||||
return None
|
||||
return self._extend_match_box(det.match_box, loc, image.shape[:2])
|
||||
|
||||
def footprint_mask(
|
||||
self, image: NDArray[Any], *, force: bool = False, dilate: int | None = None
|
||||
self,
|
||||
image: NDArray[Any] | None,
|
||||
*,
|
||||
force: bool = False,
|
||||
dilate: int | None = None,
|
||||
detection: TextMarkDetection | None = None,
|
||||
) -> NDArray[Any] | None:
|
||||
"""Full-frame uint8 mask (255 = mark) of the mark footprint, for the shared
|
||||
fill removal path (cv2 / MI-GAN / LaMa), or None if no glyph is found.
|
||||
@@ -662,6 +807,10 @@ class TextMarkEngine:
|
||||
|
||||
With ``force`` and no glyph found, falls back to the whole geometry box (the
|
||||
``--no-detect`` path). The caller gates on detection.
|
||||
|
||||
``detection`` is the caller's already-computed detection, threaded in so the
|
||||
footprint does not re-run a sweep the detector already ran. See
|
||||
:meth:`_strict_detection` for when it is reused.
|
||||
"""
|
||||
if image is None or image.size == 0:
|
||||
return None # guard before to_bgr (cvtColor raises on an empty Mat); mirror detect()
|
||||
@@ -670,46 +819,10 @@ class TextMarkEngine:
|
||||
if h < 32 or w < 64:
|
||||
return None
|
||||
loc = self.locate(image)
|
||||
bx, by, bw, bh = loc.bbox
|
||||
glyph = self.extract_mask(image, loc) # box-sized, 255 = glyph
|
||||
ys, xs = np.where(glyph > 0)
|
||||
box: tuple[int, int, int, int] | None = None
|
||||
if self.config.detect_frontend == "gray" and self.detect(image).detected:
|
||||
# The gray front-end exists for marks the top-hat under-segments, so the
|
||||
# binary blob is NOT authoritative here: trusting it first bounded the
|
||||
# fill by a PARTIAL blob (the faint head glyphs dropped out) and left the
|
||||
# leftmost "Runni" of "RunningHub AI生成" unremoved (2026-07-22). Use the
|
||||
# detector's own best-match box, same as the tophat faint path below.
|
||||
_, box = self._gray_best(image, loc)
|
||||
elif self.config.detect_frontend == "contrast" and self.detect(image).detected:
|
||||
# A dark-on-light Yuanbao mark has no WHITE top-hat blob at all. Bound
|
||||
# the fill by the polarity-independent detector's own match box.
|
||||
_, box = self._contrast_best(image, loc)
|
||||
elif xs.size >= self._MIN_GLYPH_PIXELS:
|
||||
box = (int(xs.min()), int(ys.min()), int(xs.max()), int(ys.max()))
|
||||
elif self.config.detect_frontend == "tophat" and self.detect(image).detected:
|
||||
# A mark found only by the CONTINUOUS front-end has no binary glyph blob to
|
||||
# bound, so the mask came back empty and removal was a silent no-op while
|
||||
# `identify` still reported the mark while removal left it untouched.
|
||||
# Use the DETECTOR'S OWN best-match box: the correlation already located the
|
||||
# mark at a position and scale, and thresholding the response was a strictly
|
||||
# worse proxy for that. An earlier fix thresholded the max-normalized uint8
|
||||
# response at 0.5 -- which selects every non-zero pixel, not "half the peak" as
|
||||
# its comment claimed -- and filled ~120% of the corner box on textured frames
|
||||
# (measured: whole corner vs 58.7% for the match box, both detector-clean).
|
||||
# Gated on an actual detection: on a clean corner the box would be spurious.
|
||||
_, box = self._tophat_best(image, loc)
|
||||
if box is not None:
|
||||
gx0, gy0, gx1, gy1 = box
|
||||
pad = max(4, int(0.10 * bh))
|
||||
rx1 = max(0, bx + gx0 - pad)
|
||||
rx2 = min(w, bx + gx1 + 1 + pad)
|
||||
ry1 = max(0, by + gy0 - pad)
|
||||
ry2 = min(h, by + gy1 + 1 + pad)
|
||||
elif force:
|
||||
rx1, ry1, rx2, ry2 = bx, by, min(w, bx + bw), min(h, by + bh)
|
||||
else:
|
||||
rect = self._footprint_rect(image, loc, force=force, detection=detection)
|
||||
if rect is None:
|
||||
return None
|
||||
rx1, ry1, rx2, ry2 = rect
|
||||
if rx1 >= rx2 or ry1 >= ry2:
|
||||
return None
|
||||
# Rectangular footprint + dilation is exactly region_eraser.boxes_to_mask (the
|
||||
@@ -717,5 +830,5 @@ class TextMarkEngine:
|
||||
# zeros/fill/MORPH_ELLIPSE-dilate here.
|
||||
from remove_ai_watermarks import region_eraser
|
||||
|
||||
d = dilate if dilate is not None else max(3, int(0.02 * bw))
|
||||
d = dilate if dilate is not None else max(3, int(0.02 * loc.w))
|
||||
return region_eraser.boxes_to_mask((h, w), [(rx1, ry1, rx2 - rx1, ry2 - ry1)], dilate=d)
|
||||
|
||||
+494
-13
@@ -16,11 +16,14 @@ Imports stay lazy (inside the functions), so ``import remove_ai_watermarks`` is
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from contextlib import suppress
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
from typing import TYPE_CHECKING, Any, Literal
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
from remove_ai_watermarks.watermark_registry import Backend, Sensitivity
|
||||
@@ -40,8 +43,17 @@ def visible_provenance(source: str | Path) -> frozenset[str]:
|
||||
"""Vendor keys that the file's local metadata confirms, the evidence that drives
|
||||
the ``auto`` sensitivity (relaxing a corroborated mark's detection trust gate).
|
||||
|
||||
Mapping: a Google/Gemini C2PA issuer -> ``"gemini"``; a China-AIGC (TC260) label
|
||||
-> ``"doubao"``/``"jimeng"``; a ``samsung_genai`` marker -> ``"samsung"``.
|
||||
Mapping: a Google/Gemini C2PA issuer -> ``"gemini"``; a ``samsung_genai`` marker ->
|
||||
``"samsung"``; a China-AIGC (TC260) label -> the vendor its ``ContentProducer``
|
||||
names (``KnownMark.tc260_producer_codes``), falling back to ByteDance's two products
|
||||
when the producer is absent or unmapped.
|
||||
|
||||
The TC260 label itself says only "this is AI", so it used to relax Doubao and
|
||||
Jimeng on EVERY China-AIGC image -- including one carrying a Qwen or Kling mark,
|
||||
where relaxing the wrong pair is pure false-positive risk and the mark actually
|
||||
present never reached the relaxed gate its own ``provenance_ncc_factor`` was
|
||||
calibrated for. The producer code identifies the signing entity, so it can.
|
||||
|
||||
Best-effort: any read error yields an empty set (no relaxation). Metadata-only, so
|
||||
it never loads cv2/torch.
|
||||
"""
|
||||
@@ -52,19 +64,33 @@ def visible_provenance(source: str | Path) -> frozenset[str]:
|
||||
from remove_ai_watermarks import identify
|
||||
|
||||
rep = identify.identify(path, check_visible=False, check_invisible=False)
|
||||
signal_names = {signal.name for signal in rep.signals}
|
||||
keys: set[str] = set()
|
||||
platform = (rep.platform or "").lower()
|
||||
if "google" in platform or "gemini" in platform:
|
||||
keys.add("gemini")
|
||||
if "aigc" in signal_names:
|
||||
keys |= {"doubao", "jimeng"}
|
||||
if "samsung_genai" in signal_names:
|
||||
keys.add("samsung")
|
||||
return frozenset(keys)
|
||||
return _provenance_from_report(rep, path)
|
||||
return frozenset()
|
||||
|
||||
|
||||
def _tc260_vendors(path: Path) -> frozenset[str]:
|
||||
"""Vendor keys a TC260 label confirms, from its ``ContentProducer`` identity.
|
||||
|
||||
An absent, unreadable or unmapped producer falls back to the historical pair rather
|
||||
than to nothing: the caller has already established that the AIGC signal fired, so
|
||||
the image IS China-AIGC labelled, and dropping to no relaxation would lose the
|
||||
detections the fallback recovers today. The re-read is deliberately isolated -- a
|
||||
failure here must narrow the answer, never discard the rest of the provenance.
|
||||
"""
|
||||
import contextlib
|
||||
|
||||
from remove_ai_watermarks._internal.constants import TC260_FALLBACK_VENDORS
|
||||
|
||||
with contextlib.suppress(Exception):
|
||||
from remove_ai_watermarks.metadata import aigc_label, uscc_of
|
||||
from remove_ai_watermarks.watermark_registry import tc260_producer_vendors
|
||||
|
||||
producer = (aigc_label(path) or {}).get("ContentProducer", "")
|
||||
if producer and (vendor := tc260_producer_vendors().get(uscc_of(producer))):
|
||||
return frozenset({vendor})
|
||||
return TC260_FALLBACK_VENDORS
|
||||
|
||||
|
||||
def _load_visible_input(source: str | Path | NDArray[Any]) -> _VisibleInput:
|
||||
"""Normalize a path/array source without making the public operation stateful."""
|
||||
if not isinstance(source, (str, Path)):
|
||||
@@ -177,3 +203,458 @@ def remove_visible(
|
||||
write_noop=write_noop,
|
||||
)
|
||||
return result, removed
|
||||
|
||||
|
||||
# ── The three-stage image pipeline (visible -> invisible -> metadata) ──
|
||||
# This is the `all` / `batch` pipeline. It lived only in cli.py, written twice with
|
||||
# divergences, so a library caller could not run the flagship path at all. The CLI is
|
||||
# now a thin wrapper: it builds the options, prints the stage lines through
|
||||
# `progress`, and turns the outcome into console text and an exit code.
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class InvisibleOptions:
|
||||
"""The invisible stage's knobs, as one value instead of a dozen parameters.
|
||||
|
||||
Mirrors ``InvisibleEngine.remove_watermark``. Immutable so a batch can build it once
|
||||
and reuse it across every image while the engine itself is cached separately.
|
||||
"""
|
||||
|
||||
strength: float | None = None
|
||||
pipeline: str = "qwen-zimage"
|
||||
seed: int | None = None
|
||||
hf_token: str | None = None
|
||||
humanize: float = 0.0
|
||||
unsharp: float = 0.0
|
||||
adaptive_polish: bool | None = None
|
||||
max_resolution: int | None = None
|
||||
controlnet_scale: float = 1.0
|
||||
cpu_offload: bool = True
|
||||
tile: bool = False
|
||||
tile_size: int = 1024
|
||||
tile_overlap: int = 128
|
||||
# Scrub even when no invisible watermark is locally detectable.
|
||||
force: bool = False
|
||||
|
||||
|
||||
# What the invisible stage did. "unavailable" is the one outcome the caller must
|
||||
# surface loudly: the file looks processed but still carries the watermark.
|
||||
InvisibleOutcome = Literal["removed", "no-signal", "unavailable"]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RemoveAllResult:
|
||||
"""Outcome of :func:`remove_all` -- what each stage actually did."""
|
||||
|
||||
output: Path
|
||||
visible_label: str | None # the removed mark(s), or None when nothing fired
|
||||
invisible: InvisibleOutcome
|
||||
|
||||
|
||||
class MetadataStripIncomplete(RuntimeError):
|
||||
"""AI metadata survived the strip, so no output was produced.
|
||||
|
||||
Raised BEFORE the final write, deliberately: the contract is that an incomplete
|
||||
strip leaves nothing on disk rather than an AI-readable file the caller might ship.
|
||||
"""
|
||||
|
||||
def __init__(self, surviving: set[str]) -> None:
|
||||
self.surviving = surviving
|
||||
super().__init__(f"AI metadata survived the strip: {', '.join(sorted(surviving))}")
|
||||
|
||||
|
||||
class _SourceEvidence:
|
||||
"""One metadata extraction per source file, serving every stage of one call.
|
||||
|
||||
``remove_all`` asks the same file two provenance questions -- which vendor the
|
||||
metadata confirms (for the visible pass) and whether an invisible target exists
|
||||
(for the scrub gate). Both start from the same file-backed extraction, and running
|
||||
them independently paid for it twice.
|
||||
|
||||
Per-CALL only, never module-level: the pipeline REWRITES its input in place on some
|
||||
paths (``batch`` with the output directory equal to the input), so a holder that
|
||||
outlived one call would answer from pre-write evidence. The individual metadata
|
||||
probes are memoized on content, which is what makes that safe -- this holder only
|
||||
removes the remaining assembly work.
|
||||
|
||||
Every accessor fails safe exactly as the function it replaces does: an extraction or
|
||||
verdict error yields no provenance (no relaxation) and an invisible target of True
|
||||
(scrub rather than skip).
|
||||
"""
|
||||
|
||||
__slots__ = ("_evidence", "_extracted", "_path")
|
||||
|
||||
def __init__(self, path: Path) -> None:
|
||||
self._path = path
|
||||
self._extracted = False
|
||||
self._evidence: Any | None = None
|
||||
|
||||
def _extract(self) -> Any | None:
|
||||
if not self._extracted:
|
||||
self._extracted = True
|
||||
with suppress(Exception):
|
||||
from remove_ai_watermarks.identify import extract_provenance_evidence
|
||||
|
||||
self._evidence = extract_provenance_evidence(self._path)
|
||||
return self._evidence
|
||||
|
||||
def visible_provenance(self) -> frozenset[str]:
|
||||
"""Vendor keys the metadata confirms; empty on any failure (no relaxation)."""
|
||||
evidence = self._extract()
|
||||
if evidence is None:
|
||||
return frozenset()
|
||||
# The suppress spans the VERDICT and the mapping, not just the extraction --
|
||||
# `visible_provenance` fails safe as a whole, and a raise here would escape as a
|
||||
# traceback where the old code returned an empty set.
|
||||
with suppress(Exception):
|
||||
from remove_ai_watermarks.identify import identify_from_evidence
|
||||
|
||||
report = identify_from_evidence(evidence)
|
||||
return _provenance_from_report(report, self._path)
|
||||
return frozenset()
|
||||
|
||||
def has_invisible_target(self) -> bool:
|
||||
"""Whether a diffusion target exists. True on any failure -- see the fail-safe
|
||||
note on ``identify.has_invisible_target``: leaving a watermark on a paid removal
|
||||
is worse than over-regenerating a clean image."""
|
||||
evidence = self._extract()
|
||||
if evidence is None:
|
||||
return True
|
||||
try:
|
||||
from remove_ai_watermarks.identify import identify_from_evidence
|
||||
|
||||
report = identify_from_evidence(evidence, image_path=self._path, check_invisible=True)
|
||||
except Exception:
|
||||
return True
|
||||
return bool(report.ai_from_metadata)
|
||||
|
||||
|
||||
def _provenance_from_report(report: Any, path: Path) -> frozenset[str]:
|
||||
"""Map an already-built report to the vendor keys it confirms.
|
||||
|
||||
Shared by :func:`visible_provenance` and the evidence holder so the two cannot
|
||||
drift; which evidence confirms which mark is data on the registry row.
|
||||
"""
|
||||
from remove_ai_watermarks.watermark_registry import known_marks
|
||||
|
||||
signal_names = {signal.name for signal in report.signals}
|
||||
platform = (report.platform or "").lower()
|
||||
keys: set[str] = set()
|
||||
aigc = False
|
||||
for mark in known_marks():
|
||||
if any(token in platform for token in mark.provenance_platform_tokens):
|
||||
keys.add(mark.key)
|
||||
if "aigc" in mark.provenance_signals and "aigc" in signal_names:
|
||||
aigc = True
|
||||
elif any(name in signal_names for name in mark.provenance_signals):
|
||||
keys.add(mark.key)
|
||||
if aigc:
|
||||
# The TC260 label is vendor-agnostic: which of its marks it confirms comes from
|
||||
# the producer identity, not from the signal firing.
|
||||
keys |= _tc260_vendors(path)
|
||||
return frozenset(keys)
|
||||
|
||||
|
||||
def remove_all(
|
||||
source: str | Path,
|
||||
output: str | Path,
|
||||
*,
|
||||
backend: Backend = "auto",
|
||||
sensitivity: Sensitivity = "auto",
|
||||
invisible: InvisibleOptions | None = None,
|
||||
engine: Any | None = None,
|
||||
progress: Callable[[str, str], None] | None = None,
|
||||
) -> RemoveAllResult:
|
||||
"""Remove visible marks, the invisible watermark, and AI metadata, in that order.
|
||||
|
||||
Stages are chained through a file in the SYSTEM temp dir, not next to ``output``:
|
||||
the point of staging is that the user never sees a partial output file during a long
|
||||
model download, and writing the partial next to the final defeats that.
|
||||
|
||||
``engine`` accepts an already-constructed ``InvisibleEngine`` so a batch can build
|
||||
the model once; leave it None to construct one per call.
|
||||
|
||||
``progress`` receives ``(stage, detail)`` per step -- ``stage`` is one of
|
||||
``visible`` / ``invisible`` / ``metadata`` and ``detail`` is a stable token, not
|
||||
prose: the caller owns the wording. The invisible stage reports its
|
||||
:data:`InvisibleOutcome`, plus a ``strength=<value>`` line before it runs.
|
||||
|
||||
Raises :class:`MetadataStripIncomplete` before writing anything when AI metadata
|
||||
survives, and ``OSError`` when the output cannot be written.
|
||||
"""
|
||||
import os
|
||||
import tempfile
|
||||
|
||||
from remove_ai_watermarks import image_io, watermark_registry
|
||||
from remove_ai_watermarks.metadata import strip_and_verify
|
||||
|
||||
def say(stage: str, detail: str) -> None:
|
||||
if progress is not None:
|
||||
progress(stage, detail)
|
||||
|
||||
opts = invisible if invisible is not None else InvisibleOptions()
|
||||
src, out = Path(source), Path(output)
|
||||
watermark_registry.validate_sensitivity(sensitivity)
|
||||
|
||||
image, alpha = image_io.read_bgr_and_alpha(src)
|
||||
if image is None:
|
||||
raise ValueError(f"Could not read image: {src}")
|
||||
|
||||
# One metadata extraction for the whole pipeline: the visible pass asks which
|
||||
# vendor is confirmed and the scrub gate asks whether a target exists, and both
|
||||
# start from the same evidence.
|
||||
evidence = _SourceEvidence(src)
|
||||
|
||||
tmp_fd, tmp_name = tempfile.mkstemp(suffix=src.suffix)
|
||||
os.close(tmp_fd)
|
||||
staged = Path(tmp_name)
|
||||
try:
|
||||
# ── 1. Visible marks ──
|
||||
result, removed = watermark_registry.remove_auto_marks(
|
||||
image,
|
||||
sensitivity=sensitivity,
|
||||
provenance=evidence.visible_provenance(),
|
||||
backend=backend,
|
||||
)
|
||||
visible_label = ", ".join(removed) if removed else None
|
||||
say("visible", visible_label or "")
|
||||
if not image_io.write_bgr_with_alpha(staged, result, alpha):
|
||||
raise OSError(f"failed to write the staged intermediate: {staged}")
|
||||
|
||||
# ── 2. Invisible watermark ──
|
||||
outcome = _run_invisible(src, staged, staged, opts, engine, say, evidence)
|
||||
|
||||
# ── 3. AI metadata ──
|
||||
# Read the pristine ORIGINAL for provenance above and the STAGED file here:
|
||||
# the visible pass has already dropped this file's C2PA.
|
||||
_, leftover = strip_and_verify(staged, staged)
|
||||
if leftover:
|
||||
# Before the write, on purpose -- see MetadataStripIncomplete.
|
||||
raise MetadataStripIncomplete(set(leftover))
|
||||
say("metadata", "stripped")
|
||||
|
||||
# The invisible stage (and the cv2.IMREAD_COLOR paths under it) drops alpha, so
|
||||
# re-attach the ORIGINAL alpha plane unchanged for transparent formats.
|
||||
final_bgr, _ = image_io.read_bgr_and_alpha(staged)
|
||||
if final_bgr is None:
|
||||
raise OSError(f"failed to read back the staged intermediate: {staged}")
|
||||
out.parent.mkdir(parents=True, exist_ok=True)
|
||||
if not image_io.write_bgr_with_alpha(out, final_bgr, alpha):
|
||||
raise OSError(f"failed to write output (is the destination writable?): {out}")
|
||||
finally:
|
||||
if staged.exists():
|
||||
staged.unlink()
|
||||
return RemoveAllResult(out, visible_label, outcome)
|
||||
|
||||
|
||||
def _run_invisible(
|
||||
vendor_source: Path,
|
||||
in_path: Path,
|
||||
out_path: Path,
|
||||
opts: InvisibleOptions,
|
||||
engine: Any | None,
|
||||
say: Callable[[str, str], None],
|
||||
evidence: _SourceEvidence,
|
||||
) -> InvisibleOutcome:
|
||||
"""Run, or deliberately skip, the diffusion scrub.
|
||||
|
||||
``vendor_source`` is the PRISTINE original: the staged/output file has already lost
|
||||
its C2PA to the visible pass, so gating or resolving the vendor from it would always
|
||||
read as "no signal" and "unknown vendor". ``in_path``/``out_path`` are what the
|
||||
engine reads and writes (the same staged file for ``remove_all``, input->output for
|
||||
an invisible-only batch).
|
||||
"""
|
||||
from remove_ai_watermarks.invisible_engine import is_available
|
||||
|
||||
if not is_available():
|
||||
say("invisible", "unavailable")
|
||||
return "unavailable"
|
||||
if not (opts.force or evidence.has_invisible_target()):
|
||||
say("invisible", "no-signal")
|
||||
return "no-signal"
|
||||
|
||||
from remove_ai_watermarks._internal.watermark_profiles import resolve_strength, vendor_for_strength
|
||||
|
||||
vendor = vendor_for_strength(vendor_source)
|
||||
# Report the strength the engine will actually execute, resolved the same way it
|
||||
# resolves it, so the reported value cannot drift from the executed one.
|
||||
with suppress(Exception):
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(vendor_source) as probe:
|
||||
say("invisible", f"strength={resolve_strength(opts.strength, vendor, opts.pipeline, size=probe.size)}")
|
||||
|
||||
if engine is None:
|
||||
from remove_ai_watermarks.invisible_engine import InvisibleEngine
|
||||
|
||||
engine = InvisibleEngine(
|
||||
pipeline=opts.pipeline,
|
||||
hf_token=opts.hf_token,
|
||||
progress_callback=lambda message: say("invisible", message),
|
||||
controlnet_conditioning_scale=opts.controlnet_scale,
|
||||
cpu_offload=opts.cpu_offload,
|
||||
)
|
||||
engine.remove_watermark(
|
||||
image_path=in_path,
|
||||
output_path=out_path,
|
||||
strength=opts.strength,
|
||||
seed=opts.seed,
|
||||
humanize=opts.humanize,
|
||||
unsharp=opts.unsharp,
|
||||
adaptive_polish=opts.adaptive_polish,
|
||||
max_resolution=opts.max_resolution,
|
||||
vendor=vendor,
|
||||
tile=opts.tile,
|
||||
tile_size=opts.tile_size,
|
||||
tile_overlap=opts.tile_overlap,
|
||||
)
|
||||
say("invisible", "removed")
|
||||
return "removed"
|
||||
|
||||
|
||||
BatchMode = Literal["all", "visible", "metadata", "invisible"]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class BatchSummary:
|
||||
"""Per-directory outcome of :func:`remove_batch`."""
|
||||
|
||||
processed: int
|
||||
failed: int
|
||||
# Files whose invisible watermark was left in place because the GPU extra is
|
||||
# missing. Non-empty means the outputs LOOK processed but still carry it.
|
||||
invisible_unavailable: list[Path]
|
||||
errors: list[tuple[Path, str]]
|
||||
|
||||
|
||||
def remove_batch(
|
||||
directory: str | Path,
|
||||
output_dir: str | Path,
|
||||
*,
|
||||
mode: BatchMode = "all",
|
||||
backend: Backend = "auto",
|
||||
sensitivity: Sensitivity = "auto",
|
||||
invisible: InvisibleOptions | None = None,
|
||||
engine: Any | None = None,
|
||||
progress: Callable[[Path, str, str], None] | None = None,
|
||||
) -> BatchSummary:
|
||||
"""Run one removal ``mode`` over every supported image in ``directory``.
|
||||
|
||||
Never raises for a single bad image: a per-file failure is counted and recorded in
|
||||
``BatchSummary.errors`` so one unreadable file cannot abandon the rest of the
|
||||
directory. ``engine`` is threaded straight through, so a caller that passes a
|
||||
constructed ``InvisibleEngine`` loads the model once for the whole run.
|
||||
|
||||
``progress`` receives ``(path, stage, detail)``. Every image ends with exactly one
|
||||
terminal stage -- ``done`` or ``failed`` -- whatever the mode does in between, so a
|
||||
caller driving a progress bar can advance on that alone.
|
||||
"""
|
||||
from remove_ai_watermarks._internal.utils import is_supported_format
|
||||
|
||||
src_dir, out_dir = Path(directory), Path(output_dir)
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
def say(path: Path, stage: str, detail: str) -> None:
|
||||
if progress is not None:
|
||||
progress(path, stage, detail)
|
||||
|
||||
processed = failed = 0
|
||||
unavailable: list[Path] = []
|
||||
errors: list[tuple[Path, str]] = []
|
||||
for img_path in sorted(p for p in src_dir.iterdir() if is_supported_format(p)):
|
||||
out_path = out_dir / img_path.name
|
||||
try:
|
||||
outcome = _run_batch_one(img_path, out_path, mode, backend, sensitivity, invisible, engine, say)
|
||||
except Exception as exc:
|
||||
failed += 1
|
||||
errors.append((img_path, str(exc)))
|
||||
say(img_path, "failed", str(exc))
|
||||
continue
|
||||
processed += 1
|
||||
if outcome == "unavailable":
|
||||
unavailable.append(img_path)
|
||||
# Exactly one terminal event per image, in every mode: a caller's progress bar
|
||||
# advances on this and nothing else. Keying it off a mode-specific stage line
|
||||
# left `visible` and `metadata` runs sitting at 0% for the whole batch.
|
||||
say(img_path, "done", outcome or "")
|
||||
return BatchSummary(processed, failed, unavailable, errors)
|
||||
|
||||
|
||||
def _run_batch_one(
|
||||
img_path: Path,
|
||||
out_path: Path,
|
||||
mode: BatchMode,
|
||||
backend: Backend,
|
||||
sensitivity: Sensitivity,
|
||||
invisible: InvisibleOptions | None,
|
||||
engine: Any | None,
|
||||
say: Callable[[Path, str, str], None],
|
||||
) -> InvisibleOutcome | None:
|
||||
"""One image, one mode. Returns the invisible outcome when that stage ran."""
|
||||
from remove_ai_watermarks import image_io
|
||||
from remove_ai_watermarks.metadata import strip_and_verify
|
||||
|
||||
if mode == "all":
|
||||
result = remove_all(
|
||||
img_path,
|
||||
out_path,
|
||||
backend=backend,
|
||||
sensitivity=sensitivity,
|
||||
invisible=invisible,
|
||||
engine=engine,
|
||||
progress=lambda stage, detail: say(img_path, stage, detail),
|
||||
)
|
||||
return result.invisible
|
||||
|
||||
# NOTE on holders in batch: `out_path` may EQUAL `img_path` (nothing forbids
|
||||
# `-o <the input directory>`), and the visible stage rewrites it. So each stage that
|
||||
# asks the file a provenance question builds its OWN holder, after any write that
|
||||
# precedes it. One holder spanning the write would answer the invisible gate from
|
||||
# pre-write evidence and scrub a file that today is correctly skipped.
|
||||
if mode == "visible":
|
||||
# Deliberately NOT `remove_visible`: its no-op branch copies the original bytes
|
||||
# through when nothing was removed, which is right for a single lossless call
|
||||
# and wrong here. A batch must produce every output through the one writer, so a
|
||||
# failed write RAISES and the run is counted and exits non-zero -- a read-only
|
||||
# output directory once produced zero files and still exited 0 (Tier E).
|
||||
# Always read the ORIGINAL: a stale out_path from a previous run must not be
|
||||
# re-processed as if it were the input.
|
||||
from remove_ai_watermarks import watermark_registry
|
||||
|
||||
image, alpha = image_io.read_bgr_and_alpha(img_path)
|
||||
if image is None:
|
||||
raise ValueError(f"Could not read image: {img_path}")
|
||||
result, _ = watermark_registry.remove_auto_marks(
|
||||
image,
|
||||
sensitivity=sensitivity,
|
||||
provenance=visible_provenance(img_path),
|
||||
backend=backend,
|
||||
)
|
||||
if not image_io.write_bgr_with_alpha(out_path, result, alpha):
|
||||
raise OSError(f"failed to write output (is the destination writable?): {out_path}")
|
||||
return None
|
||||
|
||||
if mode == "metadata":
|
||||
_, leftover = strip_and_verify(img_path, out_path)
|
||||
if leftover:
|
||||
raise MetadataStripIncomplete(set(leftover))
|
||||
return None
|
||||
|
||||
# invisible-only: no preceding visible pass, so out_path does not exist yet, and
|
||||
# the input IS the pristine original for both the gate and the vendor probe.
|
||||
outcome = _run_invisible(
|
||||
img_path,
|
||||
img_path,
|
||||
out_path,
|
||||
invisible if invisible is not None else InvisibleOptions(),
|
||||
engine,
|
||||
lambda stage, detail: say(img_path, stage, detail),
|
||||
_SourceEvidence(img_path),
|
||||
)
|
||||
if not out_path.exists():
|
||||
# Keep the output directory COMPLETE even when the pixels are deliberately
|
||||
# left alone; a hole the caller cannot see is worse than an unchanged copy.
|
||||
src_bgr, src_alpha = image_io.read_bgr_and_alpha(img_path)
|
||||
if src_bgr is None or not image_io.write_bgr_with_alpha(out_path, src_bgr, src_alpha):
|
||||
raise OSError(f"failed to copy input through to output: {out_path}")
|
||||
return outcome
|
||||
|
||||
@@ -30,11 +30,14 @@ from __future__ import annotations
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import (
|
||||
TextMarkConfig,
|
||||
TextMarkDetection,
|
||||
TextMarkEngine,
|
||||
TextMarkLocation,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
# Locate geometry as a fraction of the image SHORT side (measured basis). The
|
||||
@@ -100,79 +103,51 @@ _CONFIG = TextMarkConfig(
|
||||
provenance_ncc_factor=1.0,
|
||||
)
|
||||
|
||||
BaiduDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled Baidu alpha template (float [0,1]), or None."""
|
||||
return _text_mark_engine.load_alpha_template(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _glyph_silhouette() -> NDArray[Any] | None:
|
||||
"""Binary "百度" silhouette (255 = glyph) from the alpha map, or None."""
|
||||
return _text_mark_engine.glyph_silhouette(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
|
||||
"""TM_CCOEFF_NORMED of the Baidu glyph silhouette against ``box_mask``."""
|
||||
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
|
||||
|
||||
|
||||
class BaiduEngine(TextMarkEngine):
|
||||
"""Detect/localize the visible Baidu "百度 AI生成" mark (bottom-right; localize -> fill)."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
|
||||
def footprint_mask(
|
||||
self, image: NDArray[Any] | None, *, force: bool = False, dilate: int | None = None
|
||||
) -> NDArray[Any] | None:
|
||||
"""Full-frame mask of the WHOLE mark (text run + the pill tag to its right).
|
||||
def _footprint_rect(
|
||||
self,
|
||||
image: NDArray[Any],
|
||||
loc: TextMarkLocation,
|
||||
*,
|
||||
force: bool,
|
||||
detection: TextMarkDetection | None,
|
||||
) -> tuple[int, int, int, int] | None:
|
||||
"""Bound the fill by the detector's match box, never by the binary glyph blob.
|
||||
|
||||
The base class's blob-bbox footprint UNDERCOVERS this mark: the white tag's
|
||||
flat interior gives no top-hat response (a top-hat answers edges, not flats),
|
||||
so the blob ends at the text run and the fill leaves the tag's right half as
|
||||
a ghost (measured 2026-07-22 on the 768x1024 cohort frame: blob bbox x
|
||||
632..746 vs the tag ending ~758). The layout is measured and fixed -- the
|
||||
text run is at the left of the locate box, the tag runs to the corner -- so
|
||||
the footprint is the detector's match box extended RIGHT to the corner.
|
||||
632..746 vs the tag ending ~758).
|
||||
"""
|
||||
if image is None or image.size == 0:
|
||||
return None
|
||||
return self._match_box_rect(image, loc, force=force, detection=detection)
|
||||
|
||||
from remove_ai_watermarks import image_io, region_eraser
|
||||
def _extend_match_box(
|
||||
self, box: tuple[int, int, int, int], loc: TextMarkLocation, frame: tuple[int, int]
|
||||
) -> tuple[int, int, int, int]:
|
||||
"""Extend the match box RIGHT to the corner end of the locate box.
|
||||
|
||||
image = image_io.to_bgr(image)
|
||||
h, w = image.shape[:2]
|
||||
if h < 32 or w < 64:
|
||||
return None
|
||||
loc = self.locate(image)
|
||||
The layout is measured and fixed: the text run is at the left of the locate
|
||||
box and the tag runs to the corner, so the mark's right edge is the box's.
|
||||
"""
|
||||
gx0, gy0, _gx1, gy1 = box
|
||||
bx, by, bw, bh = loc.bbox
|
||||
if force:
|
||||
rx1, ry1, rx2, ry2 = bx, by, min(w, bx + bw), min(h, by + bh)
|
||||
else:
|
||||
if not self.detect(image).detected:
|
||||
return None
|
||||
_, box = self._tophat_best(image, loc)
|
||||
if box is None:
|
||||
return None
|
||||
gx0, gy0, _gx1, gy1 = box
|
||||
pad = max(4, int(0.15 * bh))
|
||||
rx1 = max(0, bx + gx0 - pad)
|
||||
ry1 = max(0, by + gy0 - pad)
|
||||
rx2 = min(w, bx + bw) # the tag runs to the corner end of the box
|
||||
ry2 = min(h, by + gy1 + 1 + pad)
|
||||
if rx1 >= rx2 or ry1 >= ry2:
|
||||
return None
|
||||
d = dilate if dilate is not None else max(3, int(0.02 * bw))
|
||||
return region_eraser.boxes_to_mask((h, w), [(rx1, ry1, rx2 - rx1, ry2 - ry1)], dilate=d)
|
||||
|
||||
|
||||
def load_image_bgr(path: str | Path) -> NDArray[Any]:
|
||||
"""Read an image as BGR ndarray (helper for scripts/tests)."""
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
img = image_io.imread(path)
|
||||
if img is None:
|
||||
raise FileNotFoundError(f"Failed to read image: {path}")
|
||||
return img
|
||||
h, w = frame
|
||||
pad = max(4, int(0.15 * bh))
|
||||
return (
|
||||
max(0, bx + gx0 - pad),
|
||||
max(0, by + gy0 - pad),
|
||||
min(w, bx + bw), # the tag runs to the corner end of the box
|
||||
min(h, by + gy1 + 1 + pad),
|
||||
)
|
||||
|
||||
+144
-376
@@ -14,7 +14,6 @@ import contextlib
|
||||
import json
|
||||
import logging
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any, Literal, NoReturn
|
||||
|
||||
@@ -44,6 +43,8 @@ if TYPE_CHECKING:
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
from remove_ai_watermarks.api import InvisibleOptions
|
||||
|
||||
|
||||
# ── plain-text output layer (replaces rich: no colors, no markup, no boxes) ──
|
||||
|
||||
@@ -81,7 +82,10 @@ class _Progress:
|
||||
def __enter__(self) -> _Progress:
|
||||
return self
|
||||
|
||||
def __exit__(self, *exc: object) -> bool:
|
||||
def __exit__(self, *exc: object) -> Literal[False]:
|
||||
# Literal[False], not bool: a plain `bool` tells a type checker this context
|
||||
# manager MAY suppress an exception, which makes every name bound inside a
|
||||
# `with` block conditionally bound afterwards. It never suppresses.
|
||||
return False
|
||||
|
||||
def add_task(self, *args: Any, **kwargs: Any) -> int:
|
||||
@@ -167,6 +171,23 @@ def _resolved_strength_for_display(
|
||||
return resolve_strength(strength, vendor, pipeline, size=image.size)
|
||||
|
||||
|
||||
# -o/--output is the most-repeated option in this module. The image commands and the
|
||||
# video commands differ only in the default they describe, so there are two decorators
|
||||
# rather than one -- same reason as every other shared option here: define it once so
|
||||
# the help text cannot drift between commands.
|
||||
_output_option = click.option(
|
||||
"-o", "--output", type=click.Path(path_type=Path), default=None, help="Output path (default: <source>_clean.<ext>)."
|
||||
)
|
||||
|
||||
_video_output_option = click.option(
|
||||
"-o",
|
||||
"--output",
|
||||
type=click.Path(path_type=Path),
|
||||
default=None,
|
||||
help="Output path (default: <source>_clean with the same container).",
|
||||
)
|
||||
|
||||
|
||||
# Shared option decorator for commands that run the invisible-watermark pipeline.
|
||||
# Both cmd_invisible and cmd_all expose this flag; defining it once avoids
|
||||
# copy-paste drift.
|
||||
@@ -327,38 +348,6 @@ def _visible_provenance(path: Path | None) -> frozenset[str]:
|
||||
return visible_provenance(path)
|
||||
|
||||
|
||||
def _remove_visible_auto(
|
||||
image: NDArray[Any],
|
||||
*,
|
||||
source_path: Path | None = None,
|
||||
backend: str = "auto",
|
||||
sensitivity: str = "auto",
|
||||
) -> tuple[NDArray[Any], str | None]:
|
||||
"""Remove every auto-detected visible mark via the registry (localize -> fill).
|
||||
|
||||
Routes the ``all``/``batch`` visible step through the same registry path the
|
||||
standalone ``visible`` command uses, so every registered mark is handled rather
|
||||
than only the Gemini sparkle.
|
||||
Returns ``(result, label-or-None)``; when no ``in_auto`` mark fires the image is
|
||||
returned unchanged with ``None``. ``backend`` selects the shared fill; ``sensitivity``
|
||||
controls how hard a borderline mark is trusted (auto reads metadata provenance)."""
|
||||
from remove_ai_watermarks import watermark_registry
|
||||
|
||||
bk: watermark_registry.Backend = backend # type: ignore[assignment]
|
||||
sens = _parse_sensitivity(sensitivity)
|
||||
provenance = _visible_provenance(source_path)
|
||||
try:
|
||||
result, removed = watermark_registry.remove_auto_marks(
|
||||
image, sensitivity=sens, provenance=provenance, backend=bk
|
||||
)
|
||||
except RuntimeError as e: # e.g. a selected migan/lama backend whose extra is absent
|
||||
console.print(f" Error: {e}")
|
||||
raise SystemExit(1) from e
|
||||
if not removed:
|
||||
return image, None
|
||||
return result, ", ".join(removed)
|
||||
|
||||
|
||||
def _parse_sensitivity(value: str) -> watermark_registry.Sensitivity:
|
||||
"""Map the CLI ``--sensitivity`` choice to the registry literal.
|
||||
|
||||
@@ -607,7 +596,9 @@ def _run_visible_explicit(
|
||||
t0 = time.monotonic()
|
||||
try:
|
||||
with console.status(f"Removing {chosen.label}... ({resolved_backend})"):
|
||||
result, _ = chosen.remove(image, backend=backend, provenance=relax, force=not detect)
|
||||
# Reuse the detection printed above instead of re-detecting inside remove():
|
||||
# nothing has touched `image` since, and the trust level is the same one.
|
||||
result, _ = chosen.remove(image, backend=backend, provenance=relax, force=not detect, detection=detection)
|
||||
except RuntimeError as e: # selected migan/lama backend whose extra is absent
|
||||
console.print(f" Error: {e}")
|
||||
raise SystemExit(1) from e
|
||||
@@ -629,9 +620,7 @@ def _run_visible_explicit(
|
||||
|
||||
@main.command("visible")
|
||||
@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path))
|
||||
@click.option(
|
||||
"-o", "--output", type=click.Path(path_type=Path), default=None, help="Output path (default: <source>_clean.<ext>)."
|
||||
)
|
||||
@_output_option
|
||||
@click.option("--detect/--no-detect", default=True, help="Detect watermark before removal.")
|
||||
@click.option(
|
||||
"--mark",
|
||||
@@ -713,9 +702,7 @@ def _parse_region(spec: str) -> tuple[int, int, int, int]:
|
||||
@main.command("erase")
|
||||
@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path))
|
||||
@click.option("--region", "regions", multiple=True, required=True, help="x,y,w,h box to erase (repeatable).")
|
||||
@click.option(
|
||||
"-o", "--output", type=click.Path(path_type=Path), default=None, help="Output path (default: <source>_clean.<ext>)."
|
||||
)
|
||||
@_output_option
|
||||
@click.option(
|
||||
"--backend",
|
||||
type=click.Choice(["cv2", "migan", "lama"]),
|
||||
@@ -787,9 +774,7 @@ def cmd_erase(
|
||||
# ── Invisible watermark removal ──
|
||||
@main.command("invisible")
|
||||
@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path))
|
||||
@click.option(
|
||||
"-o", "--output", type=click.Path(path_type=Path), default=None, help="Output path (default: <source>_clean.<ext>)."
|
||||
)
|
||||
@_output_option
|
||||
@_strength_option
|
||||
@_pipeline_option
|
||||
@_seed_option
|
||||
@@ -1066,13 +1051,7 @@ def cmd_video_identify(source: Path, no_visible: bool, as_json: bool) -> None:
|
||||
@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path))
|
||||
@click.option("--check", is_flag=True, help="Check for AI metadata (don't modify).")
|
||||
@click.option("--remove", is_flag=True, help="Remove AI metadata.")
|
||||
@click.option(
|
||||
"-o",
|
||||
"--output",
|
||||
type=click.Path(path_type=Path),
|
||||
default=None,
|
||||
help="Output path (default: <source>_clean with the same container).",
|
||||
)
|
||||
@_video_output_option
|
||||
@click.option("--keep-standard/--remove-all", default=True, help="Keep standard metadata.")
|
||||
def cmd_video_metadata(
|
||||
source: Path,
|
||||
@@ -1110,13 +1089,7 @@ def cmd_video_metadata(
|
||||
|
||||
@cmd_video.command("invisible")
|
||||
@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path))
|
||||
@click.option(
|
||||
"-o",
|
||||
"--output",
|
||||
type=click.Path(path_type=Path),
|
||||
default=None,
|
||||
help="Output path (default: <source>_clean with the same container).",
|
||||
)
|
||||
@_video_output_option
|
||||
@_video_invisible_options
|
||||
def cmd_video_invisible(
|
||||
source: Path,
|
||||
@@ -1158,13 +1131,7 @@ def cmd_video_invisible(
|
||||
|
||||
@cmd_video.command("visible")
|
||||
@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path))
|
||||
@click.option(
|
||||
"-o",
|
||||
"--output",
|
||||
type=click.Path(path_type=Path),
|
||||
default=None,
|
||||
help="Output path (default: <source>_clean with the same container).",
|
||||
)
|
||||
@_video_output_option
|
||||
@_video_visible_options
|
||||
@click.option("--strip-metadata/--keep-metadata", default=True, help="Strip AI metadata from the transcoded output.")
|
||||
def cmd_video_visible(
|
||||
@@ -1206,13 +1173,7 @@ def cmd_video_visible(
|
||||
|
||||
@cmd_video.command("all")
|
||||
@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path))
|
||||
@click.option(
|
||||
"-o",
|
||||
"--output",
|
||||
type=click.Path(path_type=Path),
|
||||
default=None,
|
||||
help="Output path (default: <source>_clean with the same container).",
|
||||
)
|
||||
@_video_output_option
|
||||
@_video_visible_options
|
||||
@click.option(
|
||||
"--invisible/--no-invisible",
|
||||
@@ -1427,9 +1388,7 @@ def cmd_identify(ctx: click.Context, source: Path, no_visible: bool, as_json: bo
|
||||
# ── Combined "all" mode ──
|
||||
@main.command("all")
|
||||
@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path))
|
||||
@click.option(
|
||||
"-o", "--output", type=click.Path(path_type=Path), default=None, help="Output path (default: <source>_clean.<ext>)."
|
||||
)
|
||||
@_output_option
|
||||
@_visible_backend_option
|
||||
@_visible_sensitivity_option
|
||||
@_strength_option
|
||||
@@ -1483,131 +1442,81 @@ def cmd_all(
|
||||
|
||||
t0 = time.monotonic()
|
||||
|
||||
# Tracks whether step 2 (invisible / SynthID removal) was skipped because the
|
||||
# GPU extra is missing. A skipped step 2 still produces an output file (visible
|
||||
# mark + metadata stripped), so without a loud end-of-run notice + non-zero exit
|
||||
# the user mistakes it for a clean result and ships an image that still carries
|
||||
# the invisible watermark (recurring reports: #14, #47).
|
||||
synthid_skipped = False
|
||||
from remove_ai_watermarks.api import InvisibleOptions, MetadataStripIncomplete, remove_all
|
||||
|
||||
# Use a temp file for intermediate results so the user doesn't see
|
||||
# a partial output file during long model downloads.
|
||||
import tempfile
|
||||
stage_labels = {
|
||||
"visible": "\n 1) Visible watermark removal",
|
||||
"invisible": "\n 2) Invisible watermark removal",
|
||||
"metadata": "\n 3) AI metadata stripping",
|
||||
}
|
||||
# The library reports WHAT happened as a (stage, detail) pair of stable tokens; the
|
||||
# console wording is the CLI's business. These two skips in particular carry guidance
|
||||
# a library caller does not need but a user very much does.
|
||||
stage_text = {
|
||||
("invisible", "no-signal"): (
|
||||
"Skipped (no invisible AI watermark detected; pixels left intact).\n"
|
||||
" Not a clean-image guarantee: a pixel SynthID is undetectable once its\n"
|
||||
" metadata proxy is gone. Re-run with --force to scrub regardless."
|
||||
),
|
||||
("invisible", "unavailable"): (
|
||||
f"Warning: Skipped - GPU dependencies not installed.\n Install them with: pip install {INVISIBLE_EXTRA}"
|
||||
),
|
||||
("invisible", "removed"): "Invisible watermark removed",
|
||||
("metadata", "stripped"): "AI metadata stripped",
|
||||
}
|
||||
seen: set[str] = set()
|
||||
|
||||
tmp_fd, tmp_path_str = tempfile.mkstemp(suffix=source.suffix)
|
||||
tmp_path = Path(tmp_path_str)
|
||||
try:
|
||||
import os
|
||||
|
||||
os.close(tmp_fd)
|
||||
|
||||
# ── Step 1: Visible watermark ──
|
||||
console.print("\n 1) Visible watermark removal")
|
||||
image, alpha = image_io.read_bgr_and_alpha(source)
|
||||
if image is None:
|
||||
console.print(f"Error: Failed to read image: {source}")
|
||||
raise SystemExit(1)
|
||||
|
||||
h, w = image.shape[:2]
|
||||
console.print(f" Input: {source.name} ({w}x{h})")
|
||||
|
||||
with console.status("Removing visible watermark..."):
|
||||
result, removed_label = _remove_visible_auto(
|
||||
image, source_path=source, backend=backend, sensitivity=sensitivity
|
||||
)
|
||||
if removed_label is not None:
|
||||
console.print(f" Visible watermark removed ({removed_label})")
|
||||
else:
|
||||
console.print(" Skipped (no visible watermark detected)")
|
||||
|
||||
# Save to temp file for invisible engine input (preserve alpha if present)
|
||||
image_io.write_bgr_with_alpha(tmp_path, result, alpha)
|
||||
|
||||
# ── Step 2: Invisible watermark ──
|
||||
console.print("\n 2) Invisible watermark removal")
|
||||
from remove_ai_watermarks.invisible_engine import is_available as invisible_available
|
||||
|
||||
if not invisible_available():
|
||||
synthid_skipped = True
|
||||
def progress(stage: str, detail: str) -> None:
|
||||
if stage in stage_labels and stage not in seen:
|
||||
seen.add(stage)
|
||||
console.print(stage_labels[stage])
|
||||
if (text := stage_text.get((stage, detail))) is not None:
|
||||
console.print(f" {text}")
|
||||
elif stage == "visible":
|
||||
console.print(
|
||||
" Warning: Skipped - GPU dependencies not installed.\n"
|
||||
f" Install them with: pip install {INVISIBLE_EXTRA}"
|
||||
)
|
||||
elif _should_skip_invisible_scrub(force, source):
|
||||
# No locally-detectable invisible watermark -> skip the destructive
|
||||
# regeneration (it would only degrade the image). The visible-removed
|
||||
# pixels in tmp_path are kept and step 3 still strips metadata, so this
|
||||
# is a SUCCESS (exit 0), unlike the GPU-missing skip above. Read the
|
||||
# pristine `source`, not tmp_path whose C2PA the visible pass already
|
||||
# dropped. Not a clean-image guarantee; --force overrides.
|
||||
console.print(
|
||||
" Skipped (no invisible AI watermark detected; pixels left intact).\n"
|
||||
" Not a clean-image guarantee: a pixel SynthID is undetectable once its\n"
|
||||
" metadata proxy is gone. Re-run with --force to scrub regardless."
|
||||
f" Visible watermark removed ({detail})" if detail else " Skipped (no visible watermark detected)"
|
||||
)
|
||||
elif detail.startswith("strength="):
|
||||
console.print(f" Strength: {detail.removeprefix('strength=')}")
|
||||
else:
|
||||
from remove_ai_watermarks.invisible_engine import InvisibleEngine
|
||||
console.print(f" {detail}")
|
||||
|
||||
def progress_cb(msg: str) -> None:
|
||||
console.print(f" {msg}")
|
||||
|
||||
inv_engine = InvisibleEngine(
|
||||
pipeline=pipeline,
|
||||
hf_token=hf_token,
|
||||
progress_callback=progress_cb,
|
||||
controlnet_conditioning_scale=controlnet_scale,
|
||||
cpu_offload=cpu_offload,
|
||||
)
|
||||
|
||||
# Detect the vendor from the pristine ORIGINAL (`source`); `tmp_path` has
|
||||
# already lost its C2PA to the visible-removal pass, so reading it would
|
||||
# always resolve to the unknown-vendor default.
|
||||
vendor = vendor_for_strength(source)
|
||||
console.print(f" Strength: {_resolved_strength_for_display(source, strength, vendor, pipeline)}")
|
||||
inv_engine.remove_watermark(
|
||||
image_path=tmp_path,
|
||||
output_path=tmp_path,
|
||||
try:
|
||||
outcome = remove_all(
|
||||
source,
|
||||
output,
|
||||
backend=backend, # type: ignore[arg-type]
|
||||
sensitivity=_parse_sensitivity(sensitivity),
|
||||
invisible=InvisibleOptions(
|
||||
strength=strength,
|
||||
pipeline=pipeline,
|
||||
seed=seed,
|
||||
hf_token=hf_token,
|
||||
humanize=humanize,
|
||||
unsharp=unsharp,
|
||||
adaptive_polish=adaptive_polish,
|
||||
max_resolution=max_resolution,
|
||||
vendor=vendor,
|
||||
controlnet_scale=controlnet_scale,
|
||||
cpu_offload=cpu_offload,
|
||||
tile=tile,
|
||||
tile_size=tile_size,
|
||||
tile_overlap=tile_overlap,
|
||||
)
|
||||
console.print(" Invisible watermark removed")
|
||||
|
||||
# ── Step 3: Metadata ──
|
||||
console.print("\n 3) AI metadata stripping")
|
||||
try:
|
||||
from remove_ai_watermarks.metadata import strip_and_verify
|
||||
|
||||
_, leftover = strip_and_verify(tmp_path, tmp_path)
|
||||
except Exception as e:
|
||||
console.print(f" Error: metadata strip failed: {e}")
|
||||
raise SystemExit(1) from e
|
||||
if leftover:
|
||||
console.print(f" Error: metadata stripping was incomplete; {', '.join(sorted(leftover))} survived")
|
||||
raise SystemExit(1)
|
||||
console.print(" AI metadata stripped")
|
||||
|
||||
# ── Write final result ──
|
||||
# The invisible step (and downstream cv2.IMREAD_COLOR paths) drops alpha,
|
||||
# so re-attach the original alpha plane unchanged when writing the final
|
||||
# output for transparent formats.
|
||||
final_bgr, _ = image_io.read_bgr_and_alpha(tmp_path)
|
||||
if final_bgr is None:
|
||||
console.print(f"Error: Failed to read intermediate file: {tmp_path}")
|
||||
raise SystemExit(1)
|
||||
_write_output_or_exit(output, final_bgr, alpha)
|
||||
|
||||
finally:
|
||||
# Clean up temp file if it still exists
|
||||
if tmp_path.exists():
|
||||
tmp_path.unlink()
|
||||
force=force,
|
||||
),
|
||||
progress=progress,
|
||||
)
|
||||
except MetadataStripIncomplete as e:
|
||||
console.print(f" Error: metadata stripping was incomplete; {', '.join(sorted(e.surviving))} survived")
|
||||
raise SystemExit(1) from e
|
||||
except ValueError as e:
|
||||
console.print(f"Error: {e}")
|
||||
raise SystemExit(1) from e
|
||||
except RuntimeError as e: # a selected migan/lama backend whose extra is absent
|
||||
console.print(f" Error: {e}")
|
||||
raise SystemExit(1) from e
|
||||
except OSError as e:
|
||||
console.print(f" Error: {e}")
|
||||
raise SystemExit(1) from e
|
||||
|
||||
# ── Done ──
|
||||
elapsed = time.monotonic() - t0
|
||||
@@ -1618,7 +1527,7 @@ def cmd_all(
|
||||
# the output looks processed but still carries the SynthID watermark. Make that
|
||||
# impossible to miss -- a prominent banner plus a non-zero exit so scripts and
|
||||
# batch callers can detect the incomplete run instead of trusting the file.
|
||||
if synthid_skipped:
|
||||
if outcome.invisible == "unavailable":
|
||||
console.print(
|
||||
"\n =====================================================================\n"
|
||||
" WARNING: the invisible (SynthID) watermark was NOT removed.\n"
|
||||
@@ -1634,172 +1543,26 @@ def cmd_all(
|
||||
|
||||
|
||||
# ── Batch command ──
|
||||
def _passthrough_copy(img_path: Path, out_path: Path) -> None:
|
||||
"""Copy the input's pixels through to ``out_path`` unchanged (the invisible-mode skip
|
||||
paths), so the output dir stays complete without touching the pixels."""
|
||||
src_bgr, src_alpha = image_io.read_bgr_and_alpha(img_path)
|
||||
if src_bgr is not None and not image_io.write_bgr_with_alpha(out_path, src_bgr, src_alpha):
|
||||
# The point of this copy is to keep the output dir COMPLETE. A silently-dropped
|
||||
# copy defeats that and leaves a hole the caller cannot see (Tier E, 2026-07-20).
|
||||
raise OSError(f"failed to copy input through to output: {out_path}")
|
||||
def _batch_engine(mode: str, options: InvisibleOptions) -> object | None:
|
||||
"""Build the ONE invisible engine this batch reuses, or None.
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _BatchOptions:
|
||||
"""Validated processing options shared by every image in one batch.
|
||||
|
||||
Click necessarily exposes these as individual command parameters, but the
|
||||
processing core should receive one coherent value instead of a long positional
|
||||
call. Keeping the object immutable also makes it safe to reuse while the
|
||||
batch caches model instances in ``ctx.obj``.
|
||||
Called once per run, not per image: ``--pipeline`` is a single CLI value, constant
|
||||
across the batch, so building the model here and threading it down is what keeps the
|
||||
diffusion stack from reloading for every file. Modes that never scrub get None, so
|
||||
nothing is loaded at all.
|
||||
"""
|
||||
if mode not in ("all", "invisible"):
|
||||
return None
|
||||
from remove_ai_watermarks.invisible_engine import InvisibleEngine, is_available
|
||||
|
||||
strength: float | None
|
||||
pipeline: str
|
||||
seed: int | None
|
||||
hf_token: str | None
|
||||
humanize: float
|
||||
backend: str = "auto"
|
||||
sensitivity: str = "auto"
|
||||
unsharp: float = 0.0
|
||||
max_resolution: int = 0
|
||||
controlnet_scale: float = 1.0
|
||||
# None means "the user did not choose"; the library resolves it per profile.
|
||||
adaptive_polish: bool | None = None
|
||||
tile: bool = False
|
||||
tile_size: int = 1024
|
||||
tile_overlap: int = 128
|
||||
force: bool = False
|
||||
cpu_offload: bool = False
|
||||
|
||||
|
||||
def _run_batch_invisible(
|
||||
ctx: click.Context,
|
||||
img_path: Path,
|
||||
out_path: Path,
|
||||
mode: str,
|
||||
options: _BatchOptions,
|
||||
) -> bool:
|
||||
"""Run or safely skip the invisible pass for one batch image.
|
||||
|
||||
Returns ``True`` only when a detectable target could not be processed because
|
||||
the GPU dependencies are missing. The availability probe is intentionally
|
||||
evaluated once so branching cannot observe inconsistent optional-dependency
|
||||
state.
|
||||
"""
|
||||
from remove_ai_watermarks.invisible_engine import is_available as invisible_available
|
||||
|
||||
skip_no_signal = _should_skip_invisible_scrub(options.force, img_path)
|
||||
available = invisible_available()
|
||||
if available and not skip_no_signal:
|
||||
from remove_ai_watermarks.invisible_engine import InvisibleEngine
|
||||
|
||||
# Cache the engine in ctx.obj so the batch builds it once (pipeline is a
|
||||
# single CLI value, constant across the run).
|
||||
engines = ctx.obj.setdefault("_inv_engines", {})
|
||||
if options.pipeline not in engines:
|
||||
engines[options.pipeline] = InvisibleEngine(
|
||||
pipeline=options.pipeline,
|
||||
hf_token=options.hf_token,
|
||||
controlnet_conditioning_scale=options.controlnet_scale,
|
||||
cpu_offload=options.cpu_offload,
|
||||
)
|
||||
engines[options.pipeline].remove_watermark(
|
||||
img_path if mode == "invisible" else out_path,
|
||||
out_path,
|
||||
strength=options.strength,
|
||||
seed=options.seed,
|
||||
humanize=options.humanize,
|
||||
unsharp=options.unsharp,
|
||||
adaptive_polish=options.adaptive_polish,
|
||||
max_resolution=options.max_resolution,
|
||||
tile=options.tile,
|
||||
tile_size=options.tile_size,
|
||||
tile_overlap=options.tile_overlap,
|
||||
# Detect the vendor from the pristine original (`img_path`), not the
|
||||
# visible-processed `out_path` whose C2PA is already gone.
|
||||
vendor=vendor_for_strength(img_path),
|
||||
)
|
||||
return False
|
||||
|
||||
# Invisible-only mode has no preceding visible pass to create ``out_path``.
|
||||
# Preserve a complete output directory while deliberately leaving pixels intact.
|
||||
if mode == "invisible" and not out_path.exists():
|
||||
_passthrough_copy(img_path, out_path)
|
||||
return not available and not skip_no_signal
|
||||
|
||||
|
||||
def _process_batch_image(
|
||||
ctx: click.Context,
|
||||
img_path: Path,
|
||||
out_path: Path,
|
||||
mode: str,
|
||||
options: _BatchOptions,
|
||||
) -> bool:
|
||||
"""Process a single image for batch mode.
|
||||
|
||||
Applies the requested watermark removal steps (visible, invisible,
|
||||
metadata) to *img_path* and writes the result to *out_path*.
|
||||
|
||||
Returns True if the invisible (SynthID) scrub was skipped because the GPU deps
|
||||
are missing while a signal was present -- so the batch caller can warn + exit
|
||||
non-zero, mirroring the single ``all`` command.
|
||||
|
||||
Raises:
|
||||
ValueError: If the image cannot be opened.
|
||||
"""
|
||||
saved_alpha: NDArray[Any] | None = None
|
||||
synthid_skipped = False
|
||||
|
||||
if mode in ("visible", "all"):
|
||||
# Always read the ORIGINAL source: the visible pass is the first step, so a
|
||||
# stale out_path from a previous run must not be re-processed as if it were
|
||||
# the input. (The invisible step below reads out_path for `all` -- that chain
|
||||
# is within a single run.)
|
||||
image, alpha = image_io.read_bgr_and_alpha(img_path)
|
||||
if image is None:
|
||||
raise ValueError("Failed to read image")
|
||||
|
||||
result, _ = _remove_visible_auto(
|
||||
image,
|
||||
source_path=img_path,
|
||||
backend=options.backend,
|
||||
sensitivity=options.sensitivity,
|
||||
)
|
||||
|
||||
# RAISE, never SystemExit: the batch loop catches per-image exceptions, counts
|
||||
# them and exits non-zero. Discarding this flag made a read-only output directory
|
||||
# produce ZERO files and still exit 0 -- silent data loss that also contradicted
|
||||
# the documented batch contract (Tier E, 2026-07-20).
|
||||
if not image_io.write_bgr_with_alpha(out_path, result, alpha):
|
||||
raise OSError(f"failed to write output (is the destination writable?): {out_path}")
|
||||
saved_alpha = alpha
|
||||
|
||||
if mode in ("invisible", "all"):
|
||||
# Skip the destructive regeneration when no invisible watermark is locally
|
||||
# detectable (would only degrade a clean image). Read the pristine `img_path`;
|
||||
# `out_path` may already be the visible-processed result. --force overrides.
|
||||
synthid_skipped = _run_batch_invisible(ctx, img_path, out_path, mode, options)
|
||||
|
||||
if mode in ("metadata", "all"):
|
||||
from remove_ai_watermarks.metadata import strip_and_verify
|
||||
|
||||
# Same verification the single-image command does: the fail-safe copy-through
|
||||
# would otherwise leave an AI-reading output and still exit 0, contradicting the
|
||||
# batch contract that a failed image must make the run exit non-zero.
|
||||
_, leftover = strip_and_verify(img_path if mode == "metadata" else out_path, out_path)
|
||||
if leftover:
|
||||
msg = f"AI metadata survived the strip ({', '.join(sorted(leftover))}); file could not be decoded"
|
||||
raise RuntimeError(msg)
|
||||
|
||||
# In "all" mode, the invisible step (color-only OpenCV paths) drops alpha,
|
||||
# so re-attach the cached alpha when the input had transparency.
|
||||
if mode == "all" and saved_alpha is not None:
|
||||
final_bgr, _ = image_io.read_bgr_and_alpha(out_path)
|
||||
if final_bgr is not None and not image_io.write_bgr_with_alpha(out_path, final_bgr, saved_alpha):
|
||||
raise OSError(f"failed to re-attach alpha to output: {out_path}")
|
||||
|
||||
return synthid_skipped
|
||||
if not is_available():
|
||||
return None
|
||||
return InvisibleEngine(
|
||||
pipeline=options.pipeline,
|
||||
hf_token=options.hf_token,
|
||||
controlnet_conditioning_scale=options.controlnet_scale,
|
||||
cpu_offload=options.cpu_offload,
|
||||
)
|
||||
|
||||
|
||||
@main.command("batch")
|
||||
@@ -1867,29 +1630,26 @@ def cmd_batch(
|
||||
console.print(f" Found {len(images)} images in {directory}")
|
||||
console.print(f" Output -> {output_dir}")
|
||||
console.print(f" Mode: {mode}")
|
||||
options = _BatchOptions(
|
||||
from remove_ai_watermarks.api import InvisibleOptions
|
||||
from remove_ai_watermarks.api import remove_batch as api_remove_batch
|
||||
|
||||
invisible_options = InvisibleOptions(
|
||||
strength=strength,
|
||||
pipeline=pipeline,
|
||||
seed=seed,
|
||||
hf_token=hf_token,
|
||||
humanize=humanize,
|
||||
backend=backend,
|
||||
sensitivity=sensitivity,
|
||||
unsharp=unsharp,
|
||||
adaptive_polish=adaptive_polish,
|
||||
max_resolution=max_resolution,
|
||||
controlnet_scale=controlnet_scale,
|
||||
adaptive_polish=adaptive_polish,
|
||||
cpu_offload=cpu_offload,
|
||||
tile=tile,
|
||||
tile_size=tile_size,
|
||||
tile_overlap=tile_overlap,
|
||||
force=force,
|
||||
cpu_offload=cpu_offload,
|
||||
)
|
||||
|
||||
processed = 0
|
||||
errors = 0
|
||||
synthid_skipped_count = 0
|
||||
|
||||
with Progress(
|
||||
SpinnerColumn(),
|
||||
TextColumn("[progress.description]{task.description}"),
|
||||
@@ -1899,28 +1659,36 @@ def cmd_batch(
|
||||
console=console,
|
||||
) as progress:
|
||||
task = progress.add_task("Processing...", total=len(images))
|
||||
done: set[str] = set()
|
||||
|
||||
for img_path in images:
|
||||
out_path = output_dir / img_path.name
|
||||
progress.update(task, description=f"{img_path.name}")
|
||||
def on_progress(img: Path, stage: str, detail: str) -> None:
|
||||
# `remove_batch` emits exactly one terminal stage per image in EVERY mode,
|
||||
# so the bar advances on that and never on a mode-specific line.
|
||||
progress.update(task, description=img.name)
|
||||
if stage in ("done", "failed") and img.name not in done:
|
||||
done.add(img.name)
|
||||
progress.advance(task)
|
||||
if ctx.obj.get("verbose"):
|
||||
console.print(f" {img.name}: {stage}{f' {detail}' if detail else ''}")
|
||||
|
||||
try:
|
||||
if _process_batch_image(
|
||||
ctx=ctx,
|
||||
img_path=img_path,
|
||||
out_path=out_path,
|
||||
mode=mode,
|
||||
options=options,
|
||||
):
|
||||
synthid_skipped_count += 1
|
||||
processed += 1
|
||||
summary = api_remove_batch(
|
||||
directory,
|
||||
output_dir,
|
||||
mode=mode, # type: ignore[arg-type]
|
||||
backend=backend, # type: ignore[arg-type]
|
||||
sensitivity=_parse_sensitivity(sensitivity),
|
||||
invisible=invisible_options,
|
||||
engine=_batch_engine(mode, invisible_options),
|
||||
progress=on_progress,
|
||||
)
|
||||
progress.update(task, completed=len(images))
|
||||
|
||||
except Exception as e:
|
||||
errors += 1
|
||||
if ctx.obj.get("verbose"):
|
||||
console.print(f" {img_path.name}: {e}")
|
||||
processed, errors = summary.processed, summary.failed
|
||||
synthid_skipped_count = len(summary.invisible_unavailable)
|
||||
|
||||
progress.advance(task)
|
||||
if errors and ctx.obj.get("verbose"):
|
||||
for failed_path, message in summary.errors:
|
||||
console.print(f" {failed_path.name}: {message}")
|
||||
|
||||
console.print(f"\n {processed} processed" + (f" {errors} errors" if errors else ""))
|
||||
|
||||
|
||||
@@ -23,11 +23,9 @@ from __future__ import annotations
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkEngine
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
# Locate geometry as a fraction of image WIDTH (the mark scales with width, anchored
|
||||
@@ -96,9 +94,6 @@ _CONFIG = TextMarkConfig(
|
||||
min_gw=8,
|
||||
)
|
||||
|
||||
# Doubao-specific aliases for the shared detection result/engine.
|
||||
DoubaoDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled Doubao alpha template (float [0,1]), or None."""
|
||||
@@ -120,13 +115,3 @@ class DoubaoEngine(TextMarkEngine):
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
|
||||
|
||||
def load_image_bgr(path: str | Path) -> NDArray[Any]:
|
||||
"""Read an image as BGR ndarray (helper for scripts/tests)."""
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
img = image_io.imread(path)
|
||||
if img is None:
|
||||
raise FileNotFoundError(f"Failed to read image: {path}")
|
||||
return img
|
||||
|
||||
@@ -6,7 +6,7 @@ from __future__ import annotations
|
||||
|
||||
import functools
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
from dataclasses import dataclass, replace
|
||||
from enum import Enum
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
@@ -72,6 +72,26 @@ class _Candidate:
|
||||
return self.spatial * 0.50 + self.gradient * 0.30 + self.variance * 0.20
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class _SparkleScan:
|
||||
"""The provenance-BLIND half of sparkle detection, reusable across trust levels.
|
||||
|
||||
``source`` is the BGR-normalized image the false-positive gate re-reads, ``best``
|
||||
the winning candidate, and ``base`` a template result carrying everything the scan
|
||||
already resolved (``size`` and, when a candidate won, ``region`` and the component
|
||||
scores). ``best is None`` covers both no-candidate cases; ``base`` distinguishes
|
||||
them, since an empty image never resolves a ``size`` and a candidate-less one does.
|
||||
|
||||
Per-CALL only, never cached on the engine: ``remove_auto_marks`` re-invokes each
|
||||
engine on a progressively cleaned frame within one process, so a memo on ``self``
|
||||
would hand back a pre-fill scan of a different image.
|
||||
"""
|
||||
|
||||
source: NDArray[Any] | None
|
||||
best: _Candidate | None
|
||||
base: DetectionResult
|
||||
|
||||
|
||||
def get_watermark_size(width: int, height: int) -> WatermarkSize:
|
||||
"""Return the provider's large tier only when both axes exceed 1024."""
|
||||
return WatermarkSize.LARGE if width > 1024 and height > 1024 else WatermarkSize.SMALL
|
||||
@@ -200,29 +220,62 @@ class GeminiEngine:
|
||||
trust_provenance: bool = False,
|
||||
) -> DetectionResult:
|
||||
"""Return the strongest sparkle-shaped bottom-right candidate."""
|
||||
result = DetectionResult()
|
||||
scan = self._sparkle_scan(image, force_size)
|
||||
return self._verdict(scan, trust_provenance=trust_provenance)
|
||||
|
||||
def detect_watermark_both(
|
||||
self, image: NDArray[Any], force_size: WatermarkSize | None = None
|
||||
) -> tuple[DetectionResult, DetectionResult]:
|
||||
"""``(strict, relaxed)`` from ONE scan of the image.
|
||||
|
||||
The scan -- global candidate search, corner promotion and fused scoring -- is
|
||||
provenance-blind; ``trust_provenance`` only decides whether the false-positive
|
||||
gate demotes the confidence afterwards. Two calls therefore repeated the whole
|
||||
sweep to reach two verdicts.
|
||||
|
||||
The two results are DISTINCT objects with genuinely different confidences (the
|
||||
gate rewrites one of them), and callers mutate them.
|
||||
"""
|
||||
scan = self._sparkle_scan(image, force_size)
|
||||
return (
|
||||
self._verdict(scan, trust_provenance=False),
|
||||
self._verdict(scan, trust_provenance=True),
|
||||
)
|
||||
|
||||
def _sparkle_scan(self, image: NDArray[Any], force_size: WatermarkSize | None) -> _SparkleScan:
|
||||
"""Everything in detection that does not depend on the trust level."""
|
||||
if image is None or image.size == 0:
|
||||
return result
|
||||
return _SparkleScan(None, None, DetectionResult())
|
||||
|
||||
source = image_io.to_bgr(image)
|
||||
height, width = source.shape[:2]
|
||||
result.size = force_size or get_watermark_size(width, height)
|
||||
size = force_size or get_watermark_size(width, height)
|
||||
candidates = self._global_candidates(source)
|
||||
promoted = self._corner_promote(source, candidates[0].spatial if candidates else -1.0)
|
||||
if promoted is not None:
|
||||
candidates.append(_Candidate(promoted[0], promoted[1], promoted[2], promoted[3]))
|
||||
# The no-candidate result is NOT the empty-image one: `size` is already resolved
|
||||
# here, and it is a public field the caller can force.
|
||||
base = DetectionResult(size=size)
|
||||
if not candidates:
|
||||
return result
|
||||
return _SparkleScan(None, None, base)
|
||||
|
||||
best = max((self._score_candidate(source, candidate) for candidate in candidates), key=lambda item: item.fused)
|
||||
result.region = (best.x, best.y, best.scale, best.scale)
|
||||
result.spatial_score = float(best.spatial)
|
||||
result.gradient_score = float(best.gradient)
|
||||
result.variance_score = float(best.variance)
|
||||
base.region = (best.x, best.y, best.scale, best.scale)
|
||||
base.spatial_score = float(best.spatial)
|
||||
base.gradient_score = float(best.gradient)
|
||||
base.variance_score = float(best.variance)
|
||||
return _SparkleScan(source, best, base)
|
||||
|
||||
def _verdict(self, scan: _SparkleScan, *, trust_provenance: bool) -> DetectionResult:
|
||||
"""Apply the trust-level-dependent tail to a scan, as a fresh result object."""
|
||||
result = replace(scan.base)
|
||||
if scan.best is None or scan.source is None:
|
||||
return result
|
||||
best = scan.best
|
||||
confidence = best.fused
|
||||
if best.spatial >= 0.25 and confidence < self._SPARKLE_FP_CONF and not trust_provenance:
|
||||
confidence = self._apply_false_positive_gate(source, best, confidence)
|
||||
confidence = self._apply_false_positive_gate(scan.source, best, confidence)
|
||||
result.confidence = float(np.clip(confidence, 0.0, 1.0))
|
||||
result.detected = result.confidence >= 0.35
|
||||
return result
|
||||
|
||||
@@ -59,6 +59,7 @@ from remove_ai_watermarks.metadata import (
|
||||
xai_signature_pair,
|
||||
)
|
||||
from remove_ai_watermarks.watermark_registry import GEMINI_SPARKLE_TRUST_CONF
|
||||
from remove_ai_watermarks.watermark_registry import known_marks as _known_marks
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
@@ -446,14 +447,22 @@ _DEVICE_C2PA_PLATFORM: tuple[tuple[bytes, str], ...] = (
|
||||
)
|
||||
|
||||
|
||||
def _device_platform(head: bytes) -> str | None:
|
||||
"""Map a distinctive C2PA device/camera token in the manifest bytes to a platform."""
|
||||
for token, platform in _DEVICE_C2PA_PLATFORM:
|
||||
def _first_token_match(head: bytes, table: tuple[tuple[bytes, str], ...]) -> str | None:
|
||||
"""First platform in ``table`` whose token appears in ``head``, else None.
|
||||
|
||||
Table order is priority: the more specific token must be listed first.
|
||||
"""
|
||||
for token, platform in table:
|
||||
if token in head:
|
||||
return platform
|
||||
return None
|
||||
|
||||
|
||||
def _device_platform(head: bytes) -> str | None:
|
||||
"""Map a distinctive C2PA device/camera token in the manifest bytes to a platform."""
|
||||
return _first_token_match(head, _DEVICE_C2PA_PLATFORM)
|
||||
|
||||
|
||||
# C2PA signers that are an editing app or AI-capable device rather than a
|
||||
# verified-capture camera. Unlike `_DEVICE_C2PA_PLATFORM`, these do NOT feed the
|
||||
# camera-vs-AI integrity clash (rule 2 in `_integrity_clashes`): a Galaxy phone
|
||||
@@ -474,10 +483,7 @@ _SIGNER_C2PA_PLATFORM: tuple[tuple[bytes, str], ...] = (
|
||||
|
||||
def _signer_platform(head: bytes) -> str | None:
|
||||
"""Map a C2PA editing-app / AI-capable-device signer token to a platform."""
|
||||
for token, platform in _SIGNER_C2PA_PLATFORM:
|
||||
if token in head:
|
||||
return platform
|
||||
return None
|
||||
return _first_token_match(head, _SIGNER_C2PA_PLATFORM)
|
||||
|
||||
|
||||
def _attribute_platform(issuers: list[str], *, is_ai: bool = True) -> str | None:
|
||||
@@ -678,16 +684,18 @@ def _visible_sparkle(image_path: Path, *, image: NDArray[Any] | None = None) ->
|
||||
# Gemini-sparkle phrasing. These are the stripped-metadata visual fallback for
|
||||
# the China-served ByteDance generators (normally also caught by the TC260 AIGC
|
||||
# metadata label); the per-engine detection thresholds live in the registry.
|
||||
_VISIBLE_MARK_PLATFORM = {
|
||||
"doubao": "ByteDance Doubao (visible 豆包AI生成 mark detected)",
|
||||
"jimeng": "ByteDance Jimeng / Dreamina (visible 即梦AI mark detected)",
|
||||
"qwen": "Alibaba Tongyi Qianwen (visible 千问AI生成 mark detected)",
|
||||
"kling": "Kuaishou Kling (visible 可灵AI 3.0 mark detected)",
|
||||
"yuanbao": "Tencent Yuanbao (visible 元宝 / AI生成 mark detected)",
|
||||
"samsung": "Samsung Galaxy AI (visible 'Contenuti generati dall'AI' mark detected)",
|
||||
"runninghub": "RunningHub (visible RunningHub AI生成 mark detected)",
|
||||
"baidu": "Baidu (visible 百度 AI生成 mark detected)",
|
||||
"liblib": "LibLibAI (visible LibLibAI mark detected)",
|
||||
# Text mark -> the platform sentence this report prints when that mark is the strongest
|
||||
# evidence, DERIVED from the registry rows so registering a mark is one edit. It was a
|
||||
# hand-maintained copy, and that class of copy is how LibLibAI ended up registered but
|
||||
# missing from the pill veto. Insertion order is the registry's, which is what fixes the
|
||||
# scan order below. The Gemini sparkle and the capture-less pill carry no platform of
|
||||
# their own (`KnownMark.platform is None`) and are excluded here: the sparkle has its
|
||||
# own higher-confidence `_visible_sparkle` path.
|
||||
#
|
||||
# Safe at module scope: `watermark_registry` is already imported above for
|
||||
# GEMINI_SPARKLE_TRUST_CONF, and it is deliberately cv2-free at import time.
|
||||
_VISIBLE_MARK_PLATFORM: dict[str, str] = {
|
||||
mark.key: mark.platform for mark in _known_marks() if mark.platform is not None
|
||||
}
|
||||
|
||||
|
||||
@@ -724,15 +732,20 @@ def _visible_text_marks(image_path: Path, *, image: NDArray[Any] | None = None)
|
||||
return detections
|
||||
|
||||
|
||||
def _invisible_watermark(image_path: Path) -> str | None:
|
||||
def _invisible_watermark(image_path: Path, decode: _SharedDecode) -> str | None:
|
||||
"""Open invisible-watermark scheme name (SD/SDXL/FLUX) or None.
|
||||
|
||||
Optional: needs the torch-free DWT-DCT decoder (extra ``detect``). Returns
|
||||
None if it is not installed or no known watermark decodes.
|
||||
"""
|
||||
from remove_ai_watermarks.invisible_watermark import detect_invisible_watermark
|
||||
from remove_ai_watermarks.invisible_watermark import detect_invisible_watermark, is_available
|
||||
|
||||
return detect_invisible_watermark(image_path)
|
||||
if not is_available():
|
||||
return None
|
||||
# `decode.get()` re-raises a decode failure exactly as the old unguarded
|
||||
# `imread` inside the detector did -- `has_invisible_target` needs that to reach
|
||||
# its fail-safe rather than silently reporting "no signal".
|
||||
return detect_invisible_watermark(image_path, image=decode.get())
|
||||
|
||||
|
||||
def _trustmark(image_path: Path) -> str | None:
|
||||
@@ -746,27 +759,79 @@ def _trustmark(image_path: Path) -> str | None:
|
||||
return detect_trustmark(image_path)
|
||||
|
||||
|
||||
class _SharedDecode:
|
||||
"""One decode of the source pixels, shared by every detector in a single report.
|
||||
|
||||
``identify`` used to decode the file three times. This holder unifies TWO of
|
||||
them -- the DWT-DCT detector and the visible-mark stage, whose own docstring
|
||||
already promised a single shared array. TrustMark keeps its own Pillow decode
|
||||
on purpose and is NOT served from here: cv2 and Pillow disagree on EXIF
|
||||
orientation and on 16-bit PNG, so feeding it this array would change what it
|
||||
decodes. An install carrying the optional ``trustmark`` extra therefore still
|
||||
pays two decodes, not one.
|
||||
|
||||
Two accessors, because the two arms need OPPOSITE failure handling:
|
||||
|
||||
* :meth:`get_or_none` swallows a decode failure and logs it. That is the visible
|
||||
arm's historical behavior -- no cv2, no visible marks, verdict unchanged.
|
||||
* :meth:`get` RE-RAISES it. The invisible arm never caught a decode error, and
|
||||
``has_invisible_target`` converts that exception into its documented fail-safe
|
||||
``True``. Swallowing it here would silently skip a diffusion scrub on a file
|
||||
that used to get one -- leaving a watermark on a paid removal.
|
||||
|
||||
Per-CALL only: constructed inside ``_identify_from_evidence`` and discarded with
|
||||
it, so an in-place rewrite between calls can never be answered from a stale array.
|
||||
"""
|
||||
|
||||
__slots__ = ("_done", "_error", "_image", "_path")
|
||||
|
||||
def __init__(self, path: Path) -> None:
|
||||
self._path = path
|
||||
self._done = False
|
||||
self._image: NDArray[Any] | None = None
|
||||
self._error: Exception | None = None
|
||||
|
||||
def _decode(self) -> None:
|
||||
if self._done:
|
||||
return
|
||||
self._done = True
|
||||
try:
|
||||
from remove_ai_watermarks.image_io import imread
|
||||
|
||||
self._image = imread(self._path)
|
||||
except Exception as exc: # cv2 missing / unreadable container
|
||||
self._error = exc
|
||||
|
||||
def get(self) -> NDArray[Any] | None:
|
||||
"""The decoded array; re-raises a decode failure, None only on a clean miss."""
|
||||
self._decode()
|
||||
if self._error is not None:
|
||||
raise self._error
|
||||
return self._image
|
||||
|
||||
def get_or_none(self) -> NDArray[Any] | None:
|
||||
"""The decoded array, or None when it could not be decoded at all."""
|
||||
self._decode()
|
||||
if self._error is not None:
|
||||
logger.debug("visible-mark decode unavailable: %s", self._error)
|
||||
return None
|
||||
return self._image
|
||||
|
||||
|
||||
def _collect_visible_signals(
|
||||
image_path: Path,
|
||||
signals: list[Signal],
|
||||
watermarks: list[str],
|
||||
platform: str | None,
|
||||
decode: _SharedDecode,
|
||||
) -> str | None:
|
||||
"""Decode once, append every trusted visible-mark signal, and return platform.
|
||||
"""Append every trusted visible-mark signal and return platform.
|
||||
|
||||
Keeping this stage separate from metadata aggregation makes the optional cv2
|
||||
boundary explicit and guarantees that all visible detectors share one decoded
|
||||
BGR array. A decode failure preserves the detectors' historical fallback/no-op
|
||||
behavior.
|
||||
All visible detectors share the one decoded BGR array held by ``decode`` (which
|
||||
the invisible detectors have usually already paid for). A decode failure
|
||||
preserves the detectors' historical fallback/no-op behavior.
|
||||
"""
|
||||
image: NDArray[Any] | None = None
|
||||
try:
|
||||
from remove_ai_watermarks.image_io import imread
|
||||
|
||||
image = imread(image_path)
|
||||
except Exception as exc: # cv2 missing - detectors fall back / no-op
|
||||
logger.debug("visible-mark decode unavailable: %s", exc)
|
||||
return platform
|
||||
image = decode.get_or_none()
|
||||
if image is None:
|
||||
return platform
|
||||
|
||||
@@ -800,6 +865,10 @@ def _identify_from_evidence(
|
||||
if (check_visible or check_invisible) and image_path is None:
|
||||
raise ValueError("Pixel-backed checks require image_path")
|
||||
pixel_path = image_path
|
||||
# One decode for every pixel detector in this report. Built here, per call, so it
|
||||
# dies with the report -- an in-place rewrite between two calls cannot be answered
|
||||
# from a stale array. Lazy inside, so a metadata-only report never decodes at all.
|
||||
decode = _SharedDecode(pixel_path) if pixel_path is not None else _SharedDecode(evidence.path)
|
||||
|
||||
info = evidence.c2pa_info
|
||||
meta = evidence.ai_metadata
|
||||
@@ -1006,7 +1075,7 @@ def _identify_from_evidence(
|
||||
|
||||
# ── Open invisible watermark (SD / SDXL / FLUX, dwtDct) ──────────
|
||||
# Public decoder, no key -- a definitive embedded signal on pristine files.
|
||||
if check_invisible and pixel_path is not None and (scheme := _invisible_watermark(pixel_path)) is not None:
|
||||
if check_invisible and pixel_path is not None and (scheme := _invisible_watermark(pixel_path, decode)) is not None:
|
||||
signals.append(Signal("invisible_watermark", scheme, "high"))
|
||||
watermarks.append(f"Open invisible watermark: {scheme}")
|
||||
caveats.append(_INVISIBLE_WM_CAVEAT)
|
||||
@@ -1039,7 +1108,7 @@ def _identify_from_evidence(
|
||||
)
|
||||
|
||||
if check_visible and pixel_path is not None:
|
||||
platform = _collect_visible_signals(pixel_path, signals, watermarks, platform)
|
||||
platform = _collect_visible_signals(pixel_path, signals, watermarks, platform, decode)
|
||||
|
||||
visible_only = any(s.name.startswith("visible_") for s in signals) and not ai_from_metadata
|
||||
hf_only = bool(hf_job) and not ai_from_metadata
|
||||
@@ -1078,9 +1147,27 @@ def _identify_from_evidence(
|
||||
)
|
||||
|
||||
|
||||
def identify_from_evidence(evidence: ProvenanceEvidence) -> ProvenanceReport:
|
||||
"""Build a metadata-only provenance verdict without reopening the source."""
|
||||
return _identify_from_evidence(evidence)
|
||||
def identify_from_evidence(
|
||||
evidence: ProvenanceEvidence,
|
||||
*,
|
||||
image_path: Path | None = None,
|
||||
check_visible: bool = False,
|
||||
check_invisible: bool = False,
|
||||
) -> ProvenanceReport:
|
||||
"""Build a provenance verdict from already-extracted evidence.
|
||||
|
||||
Metadata-only by default -- the source is never reopened. Pass ``image_path`` with
|
||||
``check_visible`` / ``check_invisible`` to add the pixel-backed detectors on top of
|
||||
the SAME evidence, which is how a caller that asks the file two provenance questions
|
||||
(which vendor is confirmed, and is there an invisible target) pays for the metadata
|
||||
extraction once.
|
||||
"""
|
||||
return _identify_from_evidence(
|
||||
evidence,
|
||||
image_path=image_path,
|
||||
check_visible=check_visible,
|
||||
check_invisible=check_invisible,
|
||||
)
|
||||
|
||||
|
||||
def identify(
|
||||
|
||||
@@ -89,6 +89,19 @@ def _pil_read(path: str | Path, flags: int) -> NDArray[Any] | None:
|
||||
return None
|
||||
|
||||
|
||||
def load_image_bgr(path: str | Path) -> NDArray[Any]:
|
||||
"""Read ``path`` as a BGR ndarray, raising instead of returning ``None``.
|
||||
|
||||
:func:`imread` keeps cv2's ``None``-on-failure contract because the removal paths
|
||||
branch on it. Scripts and tests want the opposite -- fail loudly at the read -- so
|
||||
they call this. Each vendor engine used to carry its own verbatim copy.
|
||||
"""
|
||||
image = imread(path)
|
||||
if image is None:
|
||||
raise FileNotFoundError(f"Failed to read image: {path}")
|
||||
return image
|
||||
|
||||
|
||||
def to_bgr(image: NDArray[Any]) -> NDArray[Any]:
|
||||
"""Return a 3-channel BGR view of ``image``, promoting grayscale and BGRA.
|
||||
|
||||
|
||||
@@ -29,6 +29,9 @@ from typing import TYPE_CHECKING
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterable
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -81,7 +84,7 @@ def _bits_to_bytes(bits: Iterable[object], nbytes: int) -> bytes:
|
||||
return bytes(int(value) for value in packed[:nbytes])
|
||||
|
||||
|
||||
def detect_invisible_watermark(image_path: Path) -> str | None:
|
||||
def detect_invisible_watermark(image_path: Path, *, image: NDArray[Any] | None = None) -> str | None:
|
||||
"""Return the embedding scheme name if a known open watermark is decoded.
|
||||
|
||||
Returns e.g. ``"Stable Diffusion XL"`` / ``"FLUX.2 (Black Forest Labs)"`` /
|
||||
@@ -94,7 +97,10 @@ def detect_invisible_watermark(image_path: Path) -> str | None:
|
||||
from remove_ai_watermarks import image_io
|
||||
from remove_ai_watermarks.dwt_dct import decode_dwt_dct_lengths
|
||||
|
||||
img = image_io.imread(image_path)
|
||||
# ``image`` lets a caller that has already decoded these pixels hand them in
|
||||
# (mirrors gemini_engine.detect_sparkle_confidence). The decoder only reads the
|
||||
# array -- it converts colour spaces into fresh buffers -- so no copy is needed.
|
||||
img = image if image is not None else image_io.imread(image_path)
|
||||
if img is None:
|
||||
return None
|
||||
|
||||
|
||||
@@ -24,7 +24,7 @@ from __future__ import annotations
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkEngine
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from numpy.typing import NDArray
|
||||
@@ -85,8 +85,6 @@ _CONFIG = TextMarkConfig(
|
||||
min_gw=8,
|
||||
)
|
||||
|
||||
JimengDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled Jimeng alpha template (float [0,1]), or None."""
|
||||
|
||||
@@ -52,11 +52,9 @@ from __future__ import annotations
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkEngine
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
# Locate geometry as a fraction of the image SHORT side (measured basis -- see
|
||||
@@ -114,8 +112,6 @@ _CONFIG = TextMarkConfig(
|
||||
provenance_ncc_factor=1.0,
|
||||
)
|
||||
|
||||
KlingDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled Kling alpha template (float [0,1]), or None."""
|
||||
@@ -127,23 +123,8 @@ def _glyph_silhouette() -> NDArray[Any] | None:
|
||||
return _text_mark_engine.glyph_silhouette(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
|
||||
"""TM_CCOEFF_NORMED of the Kling glyph silhouette against ``box_mask``."""
|
||||
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
|
||||
|
||||
|
||||
class KlingEngine(TextMarkEngine):
|
||||
"""Detect/localize the visible Kling "可灵AI 3.0" watermark (locate -> mask; mask feeds the fill)."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
|
||||
|
||||
def load_image_bgr(path: str | Path) -> NDArray[Any]:
|
||||
"""Read an image as BGR ndarray (helper for scripts/tests)."""
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
img = image_io.imread(path)
|
||||
if img is None:
|
||||
raise FileNotFoundError(f"Failed to read image: {path}")
|
||||
return img
|
||||
|
||||
@@ -25,11 +25,15 @@ from __future__ import annotations
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import (
|
||||
TextMarkConfig,
|
||||
TextMarkDetection,
|
||||
TextMarkEngine,
|
||||
TextMarkLocation,
|
||||
TextMarkScan,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
# Locate geometry as a fraction of the image WIDTH (measured basis). The box is
|
||||
@@ -84,24 +88,12 @@ _CONFIG = TextMarkConfig(
|
||||
provenance_ncc_factor=1.0,
|
||||
)
|
||||
|
||||
LibLibDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled LibLibAI alpha template (float [0,1]), or None."""
|
||||
return _text_mark_engine.load_alpha_template(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _glyph_silhouette() -> NDArray[Any] | None:
|
||||
"""Binary "LibLibAI" silhouette (255 = glyph) from the alpha map, or None."""
|
||||
return _text_mark_engine.glyph_silhouette(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
|
||||
"""TM_CCOEFF_NORMED of the LibLibAI glyph silhouette against ``box_mask``."""
|
||||
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
|
||||
|
||||
|
||||
class LibLibEngine(TextMarkEngine):
|
||||
"""Detect/localize the visible LibLibAI wordmark (bottom-center; localize -> fill)."""
|
||||
|
||||
@@ -111,60 +103,51 @@ class LibLibEngine(TextMarkEngine):
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
|
||||
def detect(self, image: NDArray[Any] | None, *, provenance: bool = False) -> TextMarkDetection:
|
||||
if image is None or not image.size or min(image.shape[:2]) < self._MIN_SHORT_SIDE:
|
||||
return TextMarkDetection()
|
||||
return super().detect(image, provenance=provenance)
|
||||
def _scan(self, image: NDArray[Any] | None) -> TextMarkScan:
|
||||
"""Skip the scan entirely below the size floor.
|
||||
|
||||
def footprint_mask(
|
||||
self, image: NDArray[Any] | None, *, force: bool = False, dilate: int | None = None
|
||||
) -> NDArray[Any] | None:
|
||||
"""Full-frame mask of the logo + wordmark, bounded by the detector's match box.
|
||||
Gating the SCAN rather than overriding ``detect`` is what keeps the floor on the
|
||||
single-pass perception path too, and it means a small image costs nothing.
|
||||
"""
|
||||
if image is None or not image.size or min(image.shape[:2]) < self._MIN_SHORT_SIDE:
|
||||
return TextMarkScan(None, None, 0)
|
||||
return super()._scan(image)
|
||||
|
||||
def _footprint_rect(
|
||||
self,
|
||||
image: NDArray[Any],
|
||||
loc: TextMarkLocation,
|
||||
*,
|
||||
force: bool,
|
||||
detection: TextMarkDetection | None,
|
||||
) -> tuple[int, int, int, int] | None:
|
||||
"""Bound the fill by the detector's match box, never by the binary glyph blob.
|
||||
|
||||
The base class's blob-bbox footprint is wrong in both directions here: the
|
||||
blob bleeds UP into bright background structure (on the 768x1024 cohort
|
||||
frame it reached y 931 and the fill ate the shirt's own print) and it does
|
||||
not own the triangle logo anyway. The match box bounds the wordmark exactly
|
||||
(that is what the NCC localized); the logo sits its own height to the LEFT
|
||||
of the text (measured on the cohort zoom: logo ~1.0x the glyph height, gap
|
||||
~0.3x), so the footprint is the match box extended left by ~1.3 heights.
|
||||
not own the triangle logo anyway.
|
||||
"""
|
||||
if image is None or image.size == 0:
|
||||
return None
|
||||
from remove_ai_watermarks import image_io, region_eraser
|
||||
return self._match_box_rect(image, loc, force=force, detection=detection)
|
||||
|
||||
image = image_io.to_bgr(image)
|
||||
h, w = image.shape[:2]
|
||||
if h < 32 or w < 64:
|
||||
return None
|
||||
loc = self.locate(image)
|
||||
bx, by, bw, bh = loc.bbox
|
||||
if force:
|
||||
rx1, ry1, rx2, ry2 = bx, by, min(w, bx + bw), min(h, by + bh)
|
||||
else:
|
||||
if not self.detect(image).detected:
|
||||
return None
|
||||
_, box = self._tophat_best(image, loc)
|
||||
if box is None:
|
||||
return None
|
||||
gx0, gy0, gx1, gy1 = box
|
||||
gh = gy1 - gy0 + 1
|
||||
pad = max(3, int(0.25 * gh))
|
||||
rx1 = max(0, bx + gx0 - int(1.3 * gh)) # the triangle logo, left of the text
|
||||
ry1 = max(0, by + gy0 - pad)
|
||||
rx2 = min(w, bx + gx1 + 1 + pad)
|
||||
ry2 = min(h, by + gy1 + 1 + pad)
|
||||
if rx1 >= rx2 or ry1 >= ry2:
|
||||
return None
|
||||
d = dilate if dilate is not None else max(3, int(0.02 * bw))
|
||||
return region_eraser.boxes_to_mask((h, w), [(rx1, ry1, rx2 - rx1, ry2 - ry1)], dilate=d)
|
||||
def _extend_match_box(
|
||||
self, box: tuple[int, int, int, int], loc: TextMarkLocation, frame: tuple[int, int]
|
||||
) -> tuple[int, int, int, int]:
|
||||
"""Extend the match box LEFT to take in the triangle logo.
|
||||
|
||||
|
||||
def load_image_bgr(path: str | Path) -> NDArray[Any]:
|
||||
"""Read an image as BGR ndarray (helper for scripts/tests)."""
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
img = image_io.imread(path)
|
||||
if img is None:
|
||||
raise FileNotFoundError(f"Failed to read image: {path}")
|
||||
return img
|
||||
The match box bounds the wordmark exactly (that is what the NCC localized);
|
||||
the logo sits its own height to the LEFT of the text (measured on the cohort
|
||||
zoom: logo ~1.0x the glyph height, gap ~0.3x), so the footprint is the match
|
||||
box extended left by ~1.3 heights.
|
||||
"""
|
||||
gx0, gy0, gx1, gy1 = box
|
||||
bx, by, _bw, _bh = loc.bbox
|
||||
h, w = frame
|
||||
gh = gy1 - gy0 + 1
|
||||
pad = max(3, int(0.25 * gh))
|
||||
return (
|
||||
max(0, bx + gx0 - int(1.3 * gh)), # the triangle logo, left of the text
|
||||
max(0, by + gy0 - pad),
|
||||
min(w, bx + gx1 + 1 + pad),
|
||||
min(h, by + gy1 + 1 + pad),
|
||||
)
|
||||
|
||||
@@ -15,7 +15,7 @@ import struct
|
||||
from typing import TYPE_CHECKING, Any, cast
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterable
|
||||
from collections.abc import Callable, Iterable
|
||||
from pathlib import Path
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -165,6 +165,21 @@ TC260_AIGC_FIELDS: frozenset[str] = frozenset(
|
||||
MAX_TC260_VALUE_BYTES = 1024 * 1024
|
||||
|
||||
|
||||
# A TC260 producer code is ``001`` + ``1`` + USCC(18) + a 5-digit app/product suffix,
|
||||
# so two codes sharing the USCC are the same legal entity registering different
|
||||
# products. Slicing is defensive: anything not matching the layout is returned as-is,
|
||||
# which also passes through the bare-name forms some generators write ("doubao",
|
||||
# "picwish").
|
||||
_USCC_START, _USCC_END = 4, 22
|
||||
|
||||
|
||||
def uscc_of(code: str) -> str:
|
||||
"""The 18-char Unified Social Credit Code embedded in a TC260 producer code."""
|
||||
if len(code) >= _USCC_END and code[:3] == "001":
|
||||
return code[_USCC_START:_USCC_END]
|
||||
return code
|
||||
|
||||
|
||||
def parse_tc260_aigc_json(value: bytes) -> dict[str, str] | None:
|
||||
"""Parse a bounded JSON object carrying at least one normative TC260 field."""
|
||||
if len(value) > MAX_TC260_VALUE_BYTES:
|
||||
@@ -270,6 +285,21 @@ def _png_late_metadata(image_path: Path, window: int) -> bytes:
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def _stat_key(image_path: Path) -> tuple[str, int, int] | None:
|
||||
"""Cache key identifying this file's exact CONTENT, or None when it cannot stat.
|
||||
|
||||
``(path, mtime_ns, size)`` -- size as well as mtime because an in-place rewrite
|
||||
can land inside the same mtime tick on a coarse filesystem, and this package does
|
||||
rewrite in place (``remove_ai_metadata(p, p)``, the batch output-equals-input
|
||||
case). A file it cannot stat is read uncached rather than failing.
|
||||
"""
|
||||
try:
|
||||
st = image_path.stat()
|
||||
except OSError:
|
||||
return None
|
||||
return (str(image_path), st.st_mtime_ns, st.st_size)
|
||||
|
||||
|
||||
def scan_head(image_path: Path, size: int = 1024 * 1024) -> bytes:
|
||||
"""First ``size`` bytes of the file, plus the payloads of any provenance
|
||||
metadata found beyond that window: ISOBMFF ``uuid`` / ``jumb`` boxes (seeking
|
||||
@@ -420,7 +450,7 @@ def aigc_label_from_metadata(data: bytes, candidates: tuple[str, ...] = ()) -> d
|
||||
return None
|
||||
|
||||
|
||||
def aigc_label(image_path: Path) -> dict[str, str] | None:
|
||||
def _aigc_label_impl(image_path: Path) -> dict[str, str] | None:
|
||||
"""Parse a China TC260 AI-labeling block, if present.
|
||||
|
||||
Supported serializations are:
|
||||
@@ -459,45 +489,44 @@ def aigc_label(image_path: Path) -> dict[str, str] | None:
|
||||
if isinstance(value, str) and (result := aigc_label_from_metadata(b"", (value,))):
|
||||
return result
|
||||
|
||||
# Native MP4/MOV TC260 metadata (TC260-PG-20257A): the ``AIGC`` key lives
|
||||
# in ``moov.udta.meta.keys`` and points to a raw JSON value in ``ilst``.
|
||||
# Read it through the bounded box walker so a tail ``moov`` after a large
|
||||
# ``mdat`` is found without loading or scanning the media payload.
|
||||
from remove_ai_watermarks._internal.isobmff import tc260_aigc_payloads
|
||||
|
||||
isobmff_candidates = tuple(payload.decode("utf-8", "replace") for payload in tc260_aigc_payloads(image_path))
|
||||
if result := aigc_label_from_metadata(b"", isobmff_candidates):
|
||||
return result
|
||||
|
||||
# Native MKV/WebM TC260 metadata: ``Segment.Tags.Tag.SimpleTag`` carries
|
||||
# ``TagName=AIGC`` and the raw JSON in ``TagString``. The EBML walker seeks
|
||||
# over clusters and reads only bounded metadata values.
|
||||
from remove_ai_watermarks._internal.ebml import tc260_aigc_payloads as ebml_tc260_aigc_payloads
|
||||
|
||||
ebml_candidates = tuple(payload.decode("utf-8", "replace") for payload in ebml_tc260_aigc_payloads(image_path))
|
||||
if result := aigc_label_from_metadata(b"", ebml_candidates):
|
||||
return result
|
||||
|
||||
# Native AVI and FLV TC260 metadata. Both readers walk their container
|
||||
# structures and skip media payloads instead of relying on a raw substring
|
||||
# that could collide inside compressed video.
|
||||
legacy_payloads: tuple[bytes, ...] = ()
|
||||
if image_path.suffix.lower() == ".avi":
|
||||
from remove_ai_watermarks._internal.riff import tc260_aigc_payloads as riff_tc260_aigc_payloads
|
||||
|
||||
legacy_payloads = riff_tc260_aigc_payloads(image_path)
|
||||
elif image_path.suffix.lower() == ".flv":
|
||||
from remove_ai_watermarks._internal.flv import tc260_aigc_payloads as flv_tc260_aigc_payloads
|
||||
|
||||
legacy_payloads = flv_tc260_aigc_payloads(image_path)
|
||||
legacy_candidates = tuple(payload.decode("utf-8", "replace") for payload in legacy_payloads)
|
||||
if result := aigc_label_from_metadata(b"", legacy_candidates):
|
||||
return result
|
||||
# Native container TC260 metadata. Every reader walks its own container structure
|
||||
# and skips media payloads instead of relying on a raw substring that could collide
|
||||
# inside compressed video, and every one of them SELF-GATES on its magic bytes --
|
||||
# returning () after a 4-12 byte read on anything else. So the route is content, not
|
||||
# extension: a correctly formatted AVI or FLV served under the wrong suffix used to
|
||||
# be missed, which contradicts this module's own rule elsewhere ("route on the
|
||||
# actual content format, not the extension").
|
||||
for reader in _tc260_container_readers():
|
||||
candidates = tuple(payload.decode("utf-8", "replace") for payload in reader(image_path))
|
||||
if result := aigc_label_from_metadata(b"", candidates):
|
||||
return result
|
||||
|
||||
data = scan_head(image_path)
|
||||
return aigc_label_from_metadata(data)
|
||||
|
||||
|
||||
def _tc260_container_readers() -> tuple[Callable[[Path], tuple[bytes, ...]], ...]:
|
||||
"""The native-container TC260 readers, most common first.
|
||||
|
||||
Static imports rather than ``importlib``: this module carries no pyright pragma, so
|
||||
a dynamically resolved callable would be ``Any`` and fail the strict gate. They stay
|
||||
function-local because ``isobmff`` imports this module's constants at import time.
|
||||
|
||||
* MP4/MOV -- the ``AIGC`` key in ``moov.udta.meta.keys`` points at raw JSON in
|
||||
``ilst``; the bounded box walker finds a tail ``moov`` after a large ``mdat``
|
||||
without loading the media payload.
|
||||
* MKV/WebM -- ``Segment.Tags.Tag.SimpleTag`` carries ``TagName=AIGC``.
|
||||
* AVI -- a ``LIST/INFO/AIGC`` chunk.
|
||||
* FLV -- ``script.onMetaData.AIGC``.
|
||||
"""
|
||||
from remove_ai_watermarks._internal.ebml import tc260_aigc_payloads as ebml_payloads
|
||||
from remove_ai_watermarks._internal.flv import tc260_aigc_payloads as flv_payloads
|
||||
from remove_ai_watermarks._internal.isobmff import tc260_aigc_payloads as isobmff_payloads
|
||||
from remove_ai_watermarks._internal.riff import tc260_aigc_payloads as riff_payloads
|
||||
|
||||
return (isobmff_payloads, ebml_payloads, riff_payloads, flv_payloads)
|
||||
|
||||
|
||||
# C2PA "Durable Content Credentials" manifest repositories (C2PA 2.4). When the
|
||||
# embedded manifest is stripped, an XMP ``dcterms:provenance`` URL can still point
|
||||
# at the vendor's cloud manifest store, from which the credentials are recoverable
|
||||
@@ -538,7 +567,7 @@ def c2pa_cloud_manifest(image_path: Path) -> str | None:
|
||||
return c2pa_cloud_manifest_in(scan_head(image_path, _QUICK_SCAN_BYTES))
|
||||
|
||||
|
||||
def huggingface_job(image_path: Path) -> str | None:
|
||||
def _huggingface_job_impl(image_path: Path) -> str | None:
|
||||
"""Return the HuggingFace job id if the image carries an ``hf-job-id`` PNG
|
||||
text chunk, else None.
|
||||
|
||||
@@ -599,7 +628,7 @@ def samsung_genai_in(data: bytes) -> int | None:
|
||||
return int(match.group(1)) or None
|
||||
|
||||
|
||||
def samsung_genai(image_path: Path) -> int | None:
|
||||
def _samsung_genai_impl(image_path: Path) -> int | None:
|
||||
"""Return Samsung's non-zero ``genAIType`` value if the image carries the
|
||||
Galaxy AI editing marker, else None.
|
||||
|
||||
@@ -636,7 +665,7 @@ def iptc_ai_system_in(data: bytes) -> str | None:
|
||||
return "fields present"
|
||||
|
||||
|
||||
def iptc_ai_system(image_path: Path) -> str | None:
|
||||
def _iptc_ai_system_impl(image_path: Path) -> str | None:
|
||||
"""Return an IPTC 2025.1 AI-disclosure note if the file carries those XMP
|
||||
properties, else None.
|
||||
|
||||
@@ -775,7 +804,7 @@ def _exif_text(ifd: dict[int, Any], tag: int) -> str:
|
||||
return value.decode("latin1", "replace").strip() if isinstance(value, bytes) else ""
|
||||
|
||||
|
||||
def xai_signature(image_path: Path) -> bool:
|
||||
def _xai_signature_impl(image_path: Path) -> bool:
|
||||
"""Detect xAI / Grok's EXIF provenance signature scheme.
|
||||
|
||||
Grok image downloads (Aurora model) carry no C2PA, XMP, SynthID, or IPTC --
|
||||
@@ -1372,3 +1401,87 @@ def remove_ai_metadata(
|
||||
|
||||
logger.info("Stripped AI metadata → %s", output_path)
|
||||
return output_path
|
||||
|
||||
|
||||
# ── Per-file probe memoization ──────────────────────────────────────────────
|
||||
# One ``identify`` reaches each of these twice (``get_ai_metadata`` internally, then
|
||||
# ``extract_provenance_evidence``), and each call re-walks the container -- the
|
||||
# ISOBMFF/EBML walks in ``aigc_label`` and the file-tail read in ``samsung_genai``
|
||||
# are the expensive ones. Keyed on (path, mtime_ns, size) so an in-place rewrite
|
||||
# invalidates; ``maxsize`` bounds memory to a handful of entries.
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=4)
|
||||
def _aigc_label_cached(path_str: str, _mtime_ns: int, _size: int) -> dict[str, str] | None:
|
||||
from pathlib import Path as _Path
|
||||
|
||||
return _aigc_label_impl(_Path(path_str))
|
||||
|
||||
|
||||
def aigc_label(image_path: Path) -> dict[str, str] | None:
|
||||
"""See :func:`_aigc_label_impl`; memoized per file content."""
|
||||
key = _stat_key(image_path)
|
||||
if key is None:
|
||||
return _aigc_label_impl(image_path)
|
||||
result = _aigc_label_cached(*key)
|
||||
return dict(result) if result is not None else None
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=4)
|
||||
def _huggingface_job_cached(path_str: str, _mtime_ns: int, _size: int) -> str | None:
|
||||
from pathlib import Path as _Path
|
||||
|
||||
return _huggingface_job_impl(_Path(path_str))
|
||||
|
||||
|
||||
def huggingface_job(image_path: Path) -> str | None:
|
||||
"""See :func:`_huggingface_job_impl`; memoized per file content."""
|
||||
key = _stat_key(image_path)
|
||||
if key is None:
|
||||
return _huggingface_job_impl(image_path)
|
||||
return _huggingface_job_cached(*key)
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=4)
|
||||
def _samsung_genai_cached(path_str: str, _mtime_ns: int, _size: int) -> int | None:
|
||||
from pathlib import Path as _Path
|
||||
|
||||
return _samsung_genai_impl(_Path(path_str))
|
||||
|
||||
|
||||
def samsung_genai(image_path: Path) -> int | None:
|
||||
"""See :func:`_samsung_genai_impl`; memoized per file content."""
|
||||
key = _stat_key(image_path)
|
||||
if key is None:
|
||||
return _samsung_genai_impl(image_path)
|
||||
return _samsung_genai_cached(*key)
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=4)
|
||||
def _iptc_ai_system_cached(path_str: str, _mtime_ns: int, _size: int) -> str | None:
|
||||
from pathlib import Path as _Path
|
||||
|
||||
return _iptc_ai_system_impl(_Path(path_str))
|
||||
|
||||
|
||||
def iptc_ai_system(image_path: Path) -> str | None:
|
||||
"""See :func:`_iptc_ai_system_impl`; memoized per file content."""
|
||||
key = _stat_key(image_path)
|
||||
if key is None:
|
||||
return _iptc_ai_system_impl(image_path)
|
||||
return _iptc_ai_system_cached(*key)
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=4)
|
||||
def _xai_signature_cached(path_str: str, _mtime_ns: int, _size: int) -> bool:
|
||||
from pathlib import Path as _Path
|
||||
|
||||
return _xai_signature_impl(_Path(path_str))
|
||||
|
||||
|
||||
def xai_signature(image_path: Path) -> bool:
|
||||
"""See :func:`_xai_signature_impl`; memoized per file content."""
|
||||
key = _stat_key(image_path)
|
||||
if key is None:
|
||||
return _xai_signature_impl(image_path)
|
||||
return _xai_signature_cached(*key)
|
||||
|
||||
@@ -61,6 +61,10 @@ _MASK_W, _MASK_H = 0.205, 0.115 # width of W, height of W
|
||||
# invisible inpaint. Threshold = median Sobel magnitude over the footprint box at a
|
||||
# normalized width. The reliable bottom-right wordmark arm is NOT texture-gated:
|
||||
# a wordmark-confirmed pill is removed regardless.
|
||||
#
|
||||
# Measured through the PRODUCT path (the `_keep_pill` gate), not the raw detector, by
|
||||
# ``scripts/pill_gate_audit.py`` -- the raw path bypasses the gate and reads as a
|
||||
# disaster that the shipped behaviour does not have. Re-run it when the gate changes.
|
||||
_FLAT_TEXTURE_MAX = 6.0
|
||||
|
||||
_silhouette: NDArray[Any] | None = None
|
||||
@@ -163,8 +167,11 @@ class PillEngine:
|
||||
box = self._footprint_box(image)
|
||||
if box is None:
|
||||
return None
|
||||
# Same primitive the shared fill uses, rather than a private zeros/fill copy.
|
||||
# `dilate=0` because this footprint is already generous by construction; the
|
||||
# box is clamped to the frame in _footprint_box and both origins are positive
|
||||
# fractions, so boxes_to_mask's own clamping is a no-op here.
|
||||
from remove_ai_watermarks import region_eraser
|
||||
|
||||
x0, y0, x1, y1 = box
|
||||
h, w = image.shape[:2]
|
||||
mask = np.zeros((h, w), np.uint8)
|
||||
mask[y0:y1, x0:x1] = 255
|
||||
return mask
|
||||
return region_eraser.boxes_to_mask(image.shape[:2], [(x0, y0, x1 - x0, y1 - y0)], dilate=0)
|
||||
|
||||
@@ -52,11 +52,9 @@ from __future__ import annotations
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkEngine
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
# Locate geometry as a fraction of the image SHORT side (measured basis -- see
|
||||
@@ -122,8 +120,6 @@ _CONFIG = TextMarkConfig(
|
||||
provenance_ncc_factor=1.0,
|
||||
)
|
||||
|
||||
QwenDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled Qwen alpha template (float [0,1]), or None."""
|
||||
@@ -135,23 +131,8 @@ def _glyph_silhouette() -> NDArray[Any] | None:
|
||||
return _text_mark_engine.glyph_silhouette(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
|
||||
"""TM_CCOEFF_NORMED of the Qwen glyph silhouette against ``box_mask``."""
|
||||
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
|
||||
|
||||
|
||||
class QwenEngine(TextMarkEngine):
|
||||
"""Detect/localize the visible Qwen "千问AI生成" watermark (locate -> mask; mask feeds the fill)."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
|
||||
|
||||
def load_image_bgr(path: str | Path) -> NDArray[Any]:
|
||||
"""Read an image as BGR ndarray (helper for scripts/tests)."""
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
img = image_io.imread(path)
|
||||
if img is None:
|
||||
raise FileNotFoundError(f"Failed to read image: {path}")
|
||||
return img
|
||||
|
||||
@@ -21,6 +21,10 @@ Backends:
|
||||
texture but ~200 MB model and ~4.7 GB peak RAM (too heavy for a small host).
|
||||
The model is downloaded on first use and cached by huggingface_hub; it is
|
||||
never bundled in this repo.
|
||||
|
||||
The per-backend RAM and wall-time figures above are reproduced by
|
||||
``scripts/resource_ceilings.py`` (fresh subprocess per measurement, synthetic inputs).
|
||||
Re-run it before changing any of them.
|
||||
"""
|
||||
|
||||
# cv2/numpy boundary: cv2 ships no usable type info, so strict pyright cannot know
|
||||
@@ -30,6 +34,7 @@ Backends:
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
from typing import TYPE_CHECKING, Any, Literal
|
||||
|
||||
import cv2
|
||||
@@ -48,9 +53,38 @@ _LAMA_FILE = "lama_fp32.onnx"
|
||||
_MIGAN_REPO = "andraniksargsyan/migan"
|
||||
_MIGAN_FILE = "migan.onnx"
|
||||
|
||||
# Cached onnxruntime sessions (loading is expensive; reuse across calls).
|
||||
_lama_session: object | None = None
|
||||
_migan_session: object | None = None
|
||||
# Cached onnxruntime sessions, keyed by backend name (loading is expensive; reuse
|
||||
# across calls).
|
||||
_sessions: dict[str, object] = {}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _LearnedBackend:
|
||||
"""One optional model-backed fill: its human label and the module-level names of
|
||||
its availability probe and erase function.
|
||||
|
||||
NAMES, not function objects: the fallback tests monkeypatch these by attribute on
|
||||
the module, and a table holding bound references would not see the patch.
|
||||
"""
|
||||
|
||||
label: str
|
||||
available: str
|
||||
erase: str
|
||||
|
||||
|
||||
# The learned tier, best-quality first. `resolve_backend`'s preference order and the
|
||||
# CLI's choices are separate literals by design (see FILL_BACKENDS below), but the
|
||||
# availability probe, install hint and dispatch all read this one table.
|
||||
_LEARNED_BACKENDS: dict[str, _LearnedBackend] = {
|
||||
"lama": _LearnedBackend("LaMa", "lama_available", "erase_lama"),
|
||||
"migan": _LearnedBackend("MI-GAN", "migan_available", "erase_migan"),
|
||||
}
|
||||
|
||||
# Every fill backend this module can execute, plus the caller-facing `auto`. Kept as a
|
||||
# literal tuple (not derived through an import) so `watermark_registry` and the CLI can
|
||||
# state their choices without importing this cv2-loading module at their import time --
|
||||
# a test pins the two in sync.
|
||||
FILL_BACKENDS: tuple[str, ...] = ("auto", "cv2", "migan", "lama")
|
||||
|
||||
|
||||
def boxes_to_mask(
|
||||
@@ -119,19 +153,25 @@ def lama_available() -> bool:
|
||||
return module_available("onnxruntime")
|
||||
|
||||
|
||||
def _get_lama_session() -> object:
|
||||
"""Load (once) the big-LaMa ONNX session, downloading the model on first use."""
|
||||
global _lama_session
|
||||
if _lama_session is not None:
|
||||
return _lama_session
|
||||
def _get_session(name: str, repo_id: str, filename: str, label: str) -> object:
|
||||
"""Load (once) an ONNX session, downloading the model on first use."""
|
||||
cached = _sessions.get(name)
|
||||
if cached is not None:
|
||||
return cached
|
||||
|
||||
import onnxruntime as ort
|
||||
from huggingface_hub import hf_hub_download
|
||||
|
||||
model_path = hf_hub_download(repo_id=_LAMA_REPO, filename=_LAMA_FILE)
|
||||
logger.info("Loading LaMa-ONNX model: %s", model_path)
|
||||
_lama_session = ort.InferenceSession(model_path, providers=["CPUExecutionProvider"])
|
||||
return _lama_session
|
||||
model_path = hf_hub_download(repo_id=repo_id, filename=filename)
|
||||
logger.info("Loading %s model: %s", label, model_path)
|
||||
session = ort.InferenceSession(model_path, providers=["CPUExecutionProvider"])
|
||||
_sessions[name] = session
|
||||
return session
|
||||
|
||||
|
||||
def _get_lama_session() -> object:
|
||||
"""The big-LaMa ONNX session (kept as a named seam the tests monkeypatch)."""
|
||||
return _get_session("lama", _LAMA_REPO, _LAMA_FILE, "LaMa-ONNX")
|
||||
|
||||
|
||||
def erase_lama(image_bgr: NDArray[Any], mask: NDArray[Any]) -> NDArray[Any]:
|
||||
@@ -196,25 +236,21 @@ def erase_lama(image_bgr: NDArray[Any], mask: NDArray[Any]) -> NDArray[Any]:
|
||||
|
||||
|
||||
def migan_available() -> bool:
|
||||
"""True when the optional MI-GAN backend can run (onnxruntime installed)."""
|
||||
"""True when the optional MI-GAN backend can run (onnxruntime installed).
|
||||
|
||||
Deliberately a separate function from :func:`lama_available` even though both
|
||||
currently reduce to the same onnxruntime probe: they are independent capability
|
||||
questions and `auto` resolves them separately (see
|
||||
``watermark_registry.preferred_inpaint_backend``).
|
||||
"""
|
||||
from .optional_deps import module_available
|
||||
|
||||
return module_available("onnxruntime")
|
||||
|
||||
|
||||
def _get_migan_session() -> object:
|
||||
"""Load (once) the MI-GAN ONNX session, downloading the model on first use."""
|
||||
global _migan_session
|
||||
if _migan_session is not None:
|
||||
return _migan_session
|
||||
|
||||
import onnxruntime as ort
|
||||
from huggingface_hub import hf_hub_download
|
||||
|
||||
model_path = hf_hub_download(repo_id=_MIGAN_REPO, filename=_MIGAN_FILE)
|
||||
logger.info("Loading MI-GAN ONNX model: %s", model_path)
|
||||
_migan_session = ort.InferenceSession(model_path, providers=["CPUExecutionProvider"])
|
||||
return _migan_session
|
||||
"""The MI-GAN ONNX session (kept as a named seam the tests monkeypatch)."""
|
||||
return _get_session("migan", _MIGAN_REPO, _MIGAN_FILE, "MI-GAN ONNX")
|
||||
|
||||
|
||||
def erase_migan(image_bgr: NDArray[Any], mask: NDArray[Any]) -> NDArray[Any]:
|
||||
@@ -308,16 +344,17 @@ def erase(
|
||||
if not mask.any():
|
||||
return image_bgr.copy()
|
||||
|
||||
if backend == "migan":
|
||||
if not migan_available():
|
||||
learned = _LEARNED_BACKENDS.get(backend)
|
||||
if learned is not None:
|
||||
# Probe and erase by MODULE NAME, not by the captured function object: the
|
||||
# availability probes and the erase functions are monkeypatched by name in the
|
||||
# fallback tests, and a table of bound references would not see those patches.
|
||||
if not globals()[learned.available]():
|
||||
raise RuntimeError(
|
||||
"MI-GAN backend requires onnxruntime. Install the extra: pip install 'remove-ai-watermarks[migan]'"
|
||||
f"{learned.label} backend requires onnxruntime. "
|
||||
f"Install the extra: pip install 'remove-ai-watermarks[{backend}]'"
|
||||
)
|
||||
return erase_migan(image_bgr, mask)
|
||||
if backend == "lama":
|
||||
if not lama_available():
|
||||
raise RuntimeError(
|
||||
"LaMa backend requires onnxruntime. Install the extra: pip install 'remove-ai-watermarks[lama]'"
|
||||
)
|
||||
return erase_lama(image_bgr, mask)
|
||||
return globals()[learned.erase](image_bgr, mask)
|
||||
# cv2 and anything unrecognized (including "auto", which a library caller may pass
|
||||
# straight through) degrade to the classical fill rather than raising.
|
||||
return erase_cv2(image_bgr, mask, method=cv2_method, radius=cv2_radius)
|
||||
|
||||
@@ -48,13 +48,16 @@ import logging
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import (
|
||||
TextMarkConfig,
|
||||
TextMarkDetection,
|
||||
TextMarkEngine,
|
||||
TextMarkScan,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
# Locate geometry as a fraction of the image WIDTH (the measured basis: every
|
||||
@@ -111,24 +114,12 @@ _CONFIG = TextMarkConfig(
|
||||
provenance_ncc_factor=1.0,
|
||||
)
|
||||
|
||||
RunningHubDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled RunningHub alpha template (float [0,1]), or None."""
|
||||
return _text_mark_engine.load_alpha_template(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _glyph_silhouette() -> NDArray[Any] | None:
|
||||
"""Binary "RunningHub AI生成" silhouette (255 = glyph) from the alpha map, or None."""
|
||||
return _text_mark_engine.glyph_silhouette(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
|
||||
"""TM_CCOEFF_NORMED of the RunningHub glyph silhouette against ``box_mask``."""
|
||||
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
|
||||
|
||||
|
||||
class RunningHubEngine(TextMarkEngine):
|
||||
"""Detect/localize the visible RunningHub "RunningHub AI生成" mark (top-left; localize -> fill)."""
|
||||
|
||||
@@ -140,18 +131,21 @@ class RunningHubEngine(TextMarkEngine):
|
||||
_ANCHOR_MAX_X = 0.025
|
||||
_ANCHOR_MAX_Y = 0.015
|
||||
|
||||
def detect(self, image: NDArray[Any], *, provenance: bool = False) -> TextMarkDetection:
|
||||
det = super().detect(image, provenance=provenance)
|
||||
if not det.detected:
|
||||
def _post_gate(self, det: TextMarkDetection, scan: TextMarkScan) -> TextMarkDetection:
|
||||
"""Demote a match that does not hug the top-left corner.
|
||||
|
||||
A shared post-gate rather than a ``detect`` override, so the single-pass
|
||||
perception path (``detect_both``) cannot skip it.
|
||||
"""
|
||||
if not det.detected or scan.loc is None:
|
||||
return det
|
||||
loc = self.locate(image)
|
||||
_, box = self._gray_best(image, loc)
|
||||
box = det.match_box # the sweep the scan already ran on this same loc
|
||||
if box is None:
|
||||
det.detected = False
|
||||
return det
|
||||
h, w = image.shape[:2]
|
||||
ax = (loc.x + box[0]) / w
|
||||
ay = (loc.y + box[1]) / h
|
||||
h, w = scan.frame
|
||||
ax = (scan.loc.x + box[0]) / w
|
||||
ay = (scan.loc.y + box[1]) / h
|
||||
if ax > self._ANCHOR_MAX_X or ay > self._ANCHOR_MAX_Y:
|
||||
logger.debug(
|
||||
"RunningHub detect: score %.3f but match off-anchor (x=%.3f y=%.3f); demoting.",
|
||||
@@ -161,13 +155,3 @@ class RunningHubEngine(TextMarkEngine):
|
||||
)
|
||||
det.detected = False
|
||||
return det
|
||||
|
||||
|
||||
def load_image_bgr(path: str | Path) -> NDArray[Any]:
|
||||
"""Read an image as BGR ndarray (helper for scripts/tests)."""
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
img = image_io.imread(path)
|
||||
if img is None:
|
||||
raise FileNotFoundError(f"Failed to read image: {path}")
|
||||
return img
|
||||
|
||||
@@ -28,7 +28,7 @@ from __future__ import annotations
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkEngine
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from numpy.typing import NDArray
|
||||
@@ -79,8 +79,6 @@ _CONFIG = TextMarkConfig(
|
||||
min_gw=16,
|
||||
)
|
||||
|
||||
SamsungDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled Samsung alpha template (float [0,1]), or None."""
|
||||
|
||||
@@ -229,58 +229,34 @@ def _visible_removal_plan(
|
||||
selected_scan: VideoScan,
|
||||
markers: dict[str, str],
|
||||
) -> tuple[list[tuple[int, int, int, int] | None], float, Literal["box", "veo"]]:
|
||||
"""Resolve one provider's stable frame regions and fill geometry."""
|
||||
"""Resolve one provider's stable frame regions and fill geometry.
|
||||
|
||||
Everything provider-specific -- the confidence floors, the run length, the fill
|
||||
padding and the mask style -- is data on ``VISIBLE_MARK_POLICIES``. The only thing
|
||||
left here is WHICH metadata predicate confirms which vendor, which genuinely is a
|
||||
mapping and not a tuning constant.
|
||||
"""
|
||||
from remove_ai_watermarks.video_visible import (
|
||||
VISIBLE_MARK_POLICIES,
|
||||
has_bytedance_video_provenance,
|
||||
has_sora_provenance,
|
||||
has_veo_provenance,
|
||||
stabilize_dola_localizations,
|
||||
stabilize_hailuo_localizations,
|
||||
stabilize_kling_localizations,
|
||||
stabilize_seedance_localizations,
|
||||
stabilize_sora_localizations,
|
||||
stabilize_veo_localizations,
|
||||
stabilize_localizations,
|
||||
)
|
||||
|
||||
if selected_mark == "sora":
|
||||
return (
|
||||
stabilize_sora_localizations(
|
||||
selected_scan.detections,
|
||||
provenance=has_sora_provenance(markers),
|
||||
),
|
||||
0.28,
|
||||
"box",
|
||||
)
|
||||
if selected_mark == "veo":
|
||||
return (
|
||||
stabilize_veo_localizations(
|
||||
selected_scan.detections,
|
||||
provenance=has_veo_provenance(markers),
|
||||
),
|
||||
0.18,
|
||||
"veo",
|
||||
)
|
||||
if selected_mark == "seedance":
|
||||
return (
|
||||
stabilize_seedance_localizations(
|
||||
selected_scan.detections,
|
||||
provenance=has_bytedance_video_provenance(markers),
|
||||
),
|
||||
0.0,
|
||||
"box",
|
||||
)
|
||||
if selected_mark == "dola":
|
||||
return (
|
||||
stabilize_dola_localizations(
|
||||
selected_scan.detections,
|
||||
provenance=has_bytedance_video_provenance(markers),
|
||||
),
|
||||
0.20,
|
||||
"box",
|
||||
)
|
||||
if selected_mark == "hailuo":
|
||||
return stabilize_hailuo_localizations(selected_scan.detections), 0.12, "box"
|
||||
return stabilize_kling_localizations(selected_scan.detections), 0.12, "box"
|
||||
confirms = {
|
||||
"sora": has_sora_provenance,
|
||||
"veo": has_veo_provenance,
|
||||
"seedance": has_bytedance_video_provenance,
|
||||
"dola": has_bytedance_video_provenance,
|
||||
}.get(selected_mark)
|
||||
policy = VISIBLE_MARK_POLICIES[selected_mark]
|
||||
regions = stabilize_localizations(
|
||||
selected_mark,
|
||||
selected_scan.detections,
|
||||
provenance=bool(confirms and confirms(markers)),
|
||||
)
|
||||
return regions, policy.padding_fraction, policy.mask_style
|
||||
|
||||
|
||||
def _select_stable_visible_mark(
|
||||
|
||||
@@ -402,7 +402,7 @@ def detect_sora_frame(
|
||||
"""Locate the strongest synthetic Sora-wordmark match in one frame.
|
||||
|
||||
The returned candidate is intentionally untrusted. Call
|
||||
:func:`stabilize_sora_localizations` across the full sequence before building
|
||||
:func:`stabilize_localizations` across the full sequence before building
|
||||
any removal mask.
|
||||
"""
|
||||
if image_bgr.size == 0:
|
||||
@@ -883,113 +883,133 @@ def _region_iou(left: Region, right: Region) -> float:
|
||||
return intersection / union if union > 0 else 0.0
|
||||
|
||||
|
||||
def stabilize_sora_localizations(
|
||||
detections: tuple[FrameLocalization, ...] | list[FrameLocalization],
|
||||
*,
|
||||
provenance: bool,
|
||||
) -> list[Region | None]:
|
||||
"""Accept only spatially recurring Sora candidates and bridge short dropouts.
|
||||
@dataclass(frozen=True)
|
||||
class VisibleMarkPolicy:
|
||||
"""One provider's temporal-arbiter tuning, plus the fill geometry it needs.
|
||||
|
||||
Metadata never creates a detection. It only allows a stable visual run whose
|
||||
scores remain below the strict confidence floor, which covers low-contrast
|
||||
Sora marks while clean metadata-bearing exports stay untouched.
|
||||
Every value here is MEASURED per provider; the arbiter itself
|
||||
(:func:`_stabilize_localizations`) is shared and knows nothing about providers.
|
||||
|
||||
``accepts_provenance`` is load-bearing rather than cosmetic. Hailuo and Kling have
|
||||
no metadata that could confirm them, so their rows force ``provenance=False``; that
|
||||
used to be guaranteed structurally by wrappers that took no ``provenance``
|
||||
parameter at all, and this flag is what preserves the guarantee now that one entry
|
||||
point serves every mark.
|
||||
|
||||
``padding_fraction`` and ``mask_style`` belong to the removal plan rather than the
|
||||
arbiter, but they are per-provider constants like the rest, so they live on the same
|
||||
row instead of in a parallel branch in ``video.py``.
|
||||
"""
|
||||
weak_floor = _SORA_PROVENANCE_WEAK_CONFIDENCE if provenance else _SORA_STRICT_WEAK_CONFIDENCE
|
||||
return _stabilize_localizations(
|
||||
detections,
|
||||
provenance=provenance,
|
||||
weak_floor=weak_floor,
|
||||
|
||||
weak_floor: float
|
||||
strong_floor: float
|
||||
transition_floor: float
|
||||
min_stable_frames: int
|
||||
cover_after_confirmation: bool
|
||||
padding_fraction: float
|
||||
mask_style: Literal["box", "veo"]
|
||||
anchor_iou: float | None = None
|
||||
accepts_provenance: bool = True
|
||||
# Weak floor to use when provenance confirms the vendor. None = the mark has no
|
||||
# relaxed band; metadata never creates a detection, it only lets a stable visual
|
||||
# run below the strict floor through.
|
||||
provenance_weak_floor: float | None = None
|
||||
|
||||
|
||||
VISIBLE_MARK_POLICIES: dict[str, VisibleMarkPolicy] = {
|
||||
"sora": VisibleMarkPolicy(
|
||||
weak_floor=_SORA_STRICT_WEAK_CONFIDENCE,
|
||||
provenance_weak_floor=_SORA_PROVENANCE_WEAK_CONFIDENCE,
|
||||
strong_floor=_SORA_STRONG_CONFIDENCE,
|
||||
transition_floor=0.45,
|
||||
min_stable_frames=_MIN_STABLE_FRAMES,
|
||||
cover_after_confirmation=False,
|
||||
)
|
||||
|
||||
|
||||
def stabilize_veo_localizations(
|
||||
detections: tuple[FrameLocalization, ...] | list[FrameLocalization],
|
||||
*,
|
||||
provenance: bool,
|
||||
) -> list[Region | None]:
|
||||
"""Accept temporally recurring current or legacy Veo candidates."""
|
||||
weak_floor = _VEO_PROVENANCE_WEAK_CONFIDENCE if provenance else _VEO_STRICT_WEAK_CONFIDENCE
|
||||
return _stabilize_localizations(
|
||||
detections,
|
||||
provenance=provenance,
|
||||
weak_floor=weak_floor,
|
||||
padding_fraction=0.28,
|
||||
mask_style="box",
|
||||
),
|
||||
"veo": VisibleMarkPolicy(
|
||||
weak_floor=_VEO_STRICT_WEAK_CONFIDENCE,
|
||||
provenance_weak_floor=_VEO_PROVENANCE_WEAK_CONFIDENCE,
|
||||
strong_floor=_VEO_STRONG_CONFIDENCE,
|
||||
transition_floor=0.35,
|
||||
min_stable_frames=_MIN_VEO_STABLE_FRAMES,
|
||||
cover_after_confirmation=True,
|
||||
)
|
||||
|
||||
|
||||
def stabilize_seedance_localizations(
|
||||
detections: tuple[FrameLocalization, ...] | list[FrameLocalization],
|
||||
*,
|
||||
provenance: bool,
|
||||
) -> list[Region | None]:
|
||||
"""Accept a recurring Seedance boxed-AI mark at a fixed position."""
|
||||
return _stabilize_localizations(
|
||||
detections,
|
||||
provenance=provenance,
|
||||
padding_fraction=0.18,
|
||||
mask_style="veo",
|
||||
),
|
||||
"seedance": VisibleMarkPolicy(
|
||||
# One floor at either trust level: provenance still gates the run acceptance
|
||||
# inside the arbiter, but the confidence bar does not move.
|
||||
weak_floor=_SEEDANCE_WEAK_CONFIDENCE,
|
||||
strong_floor=_SEEDANCE_STRONG_CONFIDENCE,
|
||||
transition_floor=0.30,
|
||||
min_stable_frames=_MIN_FIXED_MARK_STABLE_FRAMES,
|
||||
cover_after_confirmation=True,
|
||||
anchor_iou=0.80,
|
||||
)
|
||||
|
||||
|
||||
def stabilize_dola_localizations(
|
||||
detections: tuple[FrameLocalization, ...] | list[FrameLocalization],
|
||||
*,
|
||||
provenance: bool,
|
||||
) -> list[Region | None]:
|
||||
"""Accept a recurring Dola AI text mark at a fixed position."""
|
||||
weak_floor = _DOLA_PROVENANCE_WEAK_CONFIDENCE if provenance else _DOLA_STRICT_WEAK_CONFIDENCE
|
||||
return _stabilize_localizations(
|
||||
detections,
|
||||
provenance=provenance,
|
||||
weak_floor=weak_floor,
|
||||
padding_fraction=0.0,
|
||||
mask_style="box",
|
||||
),
|
||||
"dola": VisibleMarkPolicy(
|
||||
weak_floor=_DOLA_STRICT_WEAK_CONFIDENCE,
|
||||
provenance_weak_floor=_DOLA_PROVENANCE_WEAK_CONFIDENCE,
|
||||
strong_floor=_DOLA_STRONG_CONFIDENCE,
|
||||
transition_floor=0.40,
|
||||
min_stable_frames=_MIN_FIXED_MARK_STABLE_FRAMES,
|
||||
cover_after_confirmation=True,
|
||||
anchor_iou=0.80,
|
||||
)
|
||||
|
||||
|
||||
def stabilize_hailuo_localizations(
|
||||
detections: tuple[FrameLocalization, ...] | list[FrameLocalization],
|
||||
) -> list[Region | None]:
|
||||
"""Accept a recurring MINIMAX/Hailuo label at a fixed position."""
|
||||
return _stabilize_localizations(
|
||||
detections,
|
||||
provenance=False,
|
||||
padding_fraction=0.20,
|
||||
mask_style="box",
|
||||
),
|
||||
"hailuo": VisibleMarkPolicy(
|
||||
weak_floor=_HAILUO_WEAK_CONFIDENCE,
|
||||
strong_floor=_HAILUO_STRONG_CONFIDENCE,
|
||||
transition_floor=0.28,
|
||||
min_stable_frames=_MIN_FIXED_MARK_STABLE_FRAMES,
|
||||
cover_after_confirmation=True,
|
||||
anchor_iou=0.80,
|
||||
)
|
||||
|
||||
|
||||
def stabilize_kling_localizations(
|
||||
detections: tuple[FrameLocalization, ...] | list[FrameLocalization],
|
||||
) -> list[Region | None]:
|
||||
"""Accept a recurring versioned Kling label at a fixed position."""
|
||||
return _stabilize_localizations(
|
||||
detections,
|
||||
provenance=False,
|
||||
padding_fraction=0.12,
|
||||
mask_style="box",
|
||||
accepts_provenance=False,
|
||||
),
|
||||
"kling": VisibleMarkPolicy(
|
||||
weak_floor=_KLING_WEAK_CONFIDENCE,
|
||||
strong_floor=_KLING_STRONG_CONFIDENCE,
|
||||
transition_floor=0.30,
|
||||
min_stable_frames=_MIN_FIXED_MARK_STABLE_FRAMES,
|
||||
cover_after_confirmation=True,
|
||||
anchor_iou=0.80,
|
||||
padding_fraction=0.12,
|
||||
mask_style="box",
|
||||
accepts_provenance=False,
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def stabilize_localizations(
|
||||
mark: str,
|
||||
detections: tuple[FrameLocalization, ...] | list[FrameLocalization],
|
||||
*,
|
||||
provenance: bool = False,
|
||||
) -> list[Region | None]:
|
||||
"""Accept only spatially/temporally recurring candidates for ``mark``.
|
||||
|
||||
Metadata never creates a detection. It only allows a stable visual run whose scores
|
||||
remain below the strict confidence floor, which covers a low-contrast mark while
|
||||
clean metadata-bearing exports stay untouched. A mark whose policy sets
|
||||
``accepts_provenance=False`` ignores the argument entirely.
|
||||
"""
|
||||
policy = VISIBLE_MARK_POLICIES[mark]
|
||||
trusted = provenance and policy.accepts_provenance
|
||||
weak_floor = policy.provenance_weak_floor if (trusted and policy.provenance_weak_floor is not None) else None
|
||||
return _stabilize_localizations(
|
||||
detections,
|
||||
provenance=trusted,
|
||||
weak_floor=weak_floor if weak_floor is not None else policy.weak_floor,
|
||||
strong_floor=policy.strong_floor,
|
||||
transition_floor=policy.transition_floor,
|
||||
min_stable_frames=policy.min_stable_frames,
|
||||
cover_after_confirmation=policy.cover_after_confirmation,
|
||||
**({"anchor_iou": policy.anchor_iou} if policy.anchor_iou is not None else {}),
|
||||
)
|
||||
|
||||
|
||||
@@ -1182,8 +1202,8 @@ def _mask_for_region(
|
||||
) -> NDArray[Any]:
|
||||
height, width = frame_bgr.shape[:2]
|
||||
x, y, region_width, region_height = region
|
||||
mask = np.zeros((height, width), dtype=np.uint8)
|
||||
if mask_style == "veo" and 0.80 <= region_width / region_height <= 1.25:
|
||||
mask = np.zeros((height, width), dtype=np.uint8)
|
||||
diamond_base, _ = _veo_templates()
|
||||
diamond = cv2.resize(
|
||||
diamond_base,
|
||||
@@ -1207,13 +1227,16 @@ def _mask_for_region(
|
||||
# into the hole, recreating the mascot as a bright blob. The measured clean
|
||||
# floor on real Sora frames is a full box with roughly 0.28 mark-heights of
|
||||
# context on every side.
|
||||
# Padded rectangle + no dilation is exactly region_eraser.boxes_to_mask (the same
|
||||
# primitive the image fill uses); the padding IS the growth, so `dilate=0`.
|
||||
from remove_ai_watermarks.region_eraser import boxes_to_mask
|
||||
|
||||
padding = max(4, round(region_height * padding_fraction))
|
||||
x0 = max(0, x - padding)
|
||||
y0 = max(0, y - padding)
|
||||
x1 = min(width, x + region_width + padding)
|
||||
y1 = min(height, y + region_height + padding)
|
||||
mask[y0:y1, x0:x1] = 255
|
||||
return mask
|
||||
return boxes_to_mask(
|
||||
(height, width),
|
||||
[(x - padding, y - padding, region_width + 2 * padding, region_height + 2 * padding)],
|
||||
dilate=0,
|
||||
)
|
||||
|
||||
|
||||
def _timestamp_time_base(profile_time_base: str | None) -> Fraction:
|
||||
|
||||
@@ -33,7 +33,7 @@ Entries:
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
from dataclasses import dataclass, field
|
||||
from typing import TYPE_CHECKING, Any, Literal
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -79,24 +79,8 @@ Sensitivity = Literal["auto", "strict"]
|
||||
# A third ``assumed`` level existed for ``assume_ai`` and went with it (2026-07-19).
|
||||
Trust = Literal["strict", "confirmed"]
|
||||
|
||||
# Product family per mark, for the ``auto`` cross-mark corroboration: a confidently
|
||||
# detected mark relaxes only OTHER marks of the SAME product (different corners, one
|
||||
# product -- the Jimeng wordmark + the Jimeng pill). Doubao and Jimeng are BOTH ByteDance
|
||||
# but distinct products in the SAME bottom-right corner, so they must NOT cross-relax
|
||||
# (relaxing Doubao on a Jimeng wordmark would spuriously fire Doubao on it).
|
||||
_PRODUCT_OF: dict[str, str] = {
|
||||
"gemini": "gemini",
|
||||
"doubao": "doubao",
|
||||
"jimeng": "jimeng",
|
||||
"jimeng_pill": "jimeng", # same product as the Jimeng wordmark
|
||||
"qwen": "qwen",
|
||||
"kling": "kling",
|
||||
"yuanbao": "yuanbao",
|
||||
"samsung": "samsung",
|
||||
"runninghub": "runninghub",
|
||||
"baidu": "baidu",
|
||||
"liblib": "liblib",
|
||||
}
|
||||
# Product family per mark now lives on the registry row (``KnownMark.product``);
|
||||
# ``_PRODUCT_OF`` is derived from it right after ``_REGISTRY`` is built.
|
||||
|
||||
|
||||
# Marks whose own detection is too weak to serve as EVIDENCE for a sibling of the
|
||||
@@ -134,6 +118,11 @@ class MarkDetection:
|
||||
detected: bool
|
||||
confidence: float
|
||||
region: Region
|
||||
# The engine's OWN detection object, threaded back to this mark's mask builder so it
|
||||
# does not re-run the detector (the text-mark footprint is bounded by the ladder
|
||||
# sweep detection already ran). Opaque here: each mask adapter knows its engine's
|
||||
# type. Excluded from eq/repr so the uniform result stays comparable across engines.
|
||||
engine_detection: Any | None = field(default=None, compare=False, repr=False)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -234,12 +223,42 @@ class KnownMark:
|
||||
label: str
|
||||
location: str # usual place, human-readable ("bottom-right")
|
||||
in_auto: bool # participate in `--mark auto` scanning
|
||||
# Product family, for the `auto` cross-mark corroboration: a confidently detected
|
||||
# mark relaxes only OTHER marks of the SAME product (different corners, one product
|
||||
# -- the Jimeng wordmark + the Jimeng pill). Doubao and Jimeng are BOTH ByteDance but
|
||||
# distinct products in the SAME bottom-right corner, so they must NOT cross-relax
|
||||
# (relaxing Doubao on a Jimeng wordmark would spuriously fire Doubao on it).
|
||||
# REQUIRED, deliberately: a defaulted empty product would let two rows that forgot
|
||||
# the field corroborate each other and silently bypass the trust gate.
|
||||
product: str
|
||||
# Which provenance regime this mark's vendor labels under, or None. "tc260" means
|
||||
# the vendor stamps the China AIGC label, so a confident detection of it names a
|
||||
# DIFFERENT TC260 product than the Jimeng pill -- see _keep_pill.
|
||||
label_regime: str | None
|
||||
# The sentence `identify` reports when THIS mark is the strongest evidence. None for
|
||||
# a mark that never names a platform on its own: `gemini` has its own higher-
|
||||
# confidence sparkle path, and the capture-less pill is too weak to attribute.
|
||||
platform: str | None
|
||||
_detect: Callable[..., MarkDetection]
|
||||
_mask: Callable[..., NDArray[Any] | None]
|
||||
# Optional physical-feature probe: the mark's OWN measurements its gate needs
|
||||
# (e.g. the pill's footprint flatness), so the perception pass stays uniform and
|
||||
# does not special-case any mark. None = the mark's gate needs no extra evidence.
|
||||
_features: Callable[..., dict[str, float]] | None = None
|
||||
# Metadata signal names that confirm this mark's vendor, and platform substrings
|
||||
# that do. Both drive `api.visible_provenance`; empty means nothing confirms it.
|
||||
provenance_signals: tuple[str, ...] = ()
|
||||
provenance_platform_tokens: tuple[str, ...] = ()
|
||||
# TC260 ``ContentProducer`` identities that name THIS mark's vendor -- Unified
|
||||
# Social Credit Codes as normalized by ``metadata.uscc_of``, plus the bare product
|
||||
# names a few generators write instead. Here rather than in a separate table
|
||||
# because a newly registered TC260 mark whose codes were forgotten fails SILENTLY:
|
||||
# it falls back to relaxing ByteDance's pair on an image carrying the new mark,
|
||||
# the exact false positive the producer code exists to prevent.
|
||||
tc260_producer_codes: tuple[str, ...] = ()
|
||||
# Optional single-pass dual verdict for the arbiter's perception stage (see
|
||||
# `detect_both`). None = fall back to two `_detect` calls.
|
||||
_detect_both: Callable[..., tuple[MarkDetection, MarkDetection]] | None = None
|
||||
|
||||
def features(self, image: NDArray[Any]) -> dict[str, float]:
|
||||
"""Physical features the mark reports for the arbiter's gate (empty if none)."""
|
||||
@@ -252,11 +271,35 @@ class KnownMark:
|
||||
is trusted when provenance says the vendor is present)."""
|
||||
return self._detect(image, provenance=provenance)
|
||||
|
||||
def localize(self, image: NDArray[Any], *, provenance: bool = False, force: bool = False) -> Localization:
|
||||
def detect_both(self, image: NDArray[Any]) -> tuple[MarkDetection, MarkDetection]:
|
||||
"""``(strict, relaxed)`` from ONE pass over the image.
|
||||
|
||||
The arbiter's perception stage needs a mark's verdict at BOTH trust levels so it
|
||||
can pick per mark without re-detecting. ``provenance`` never changes what a
|
||||
detector computes -- only the threshold it compares against, or (Gemini) whether
|
||||
a false-positive gate demotes the result afterwards -- so the expensive scan is
|
||||
shared. A mark without a ``_detect_both`` adapter falls back to two calls."""
|
||||
if self._detect_both is not None:
|
||||
return self._detect_both(image)
|
||||
return self.detect(image, provenance=False), self.detect(image, provenance=True)
|
||||
|
||||
def localize(
|
||||
self,
|
||||
image: NDArray[Any],
|
||||
*,
|
||||
provenance: bool = False,
|
||||
force: bool = False,
|
||||
detection: MarkDetection | None = None,
|
||||
) -> Localization:
|
||||
"""Detect and build the removal mask in one call. Returns a
|
||||
:class:`Localization`; ``mask`` is None unless the mark is detected (or
|
||||
``force`` bypasses detection for the mark's usual footprint)."""
|
||||
det = self.detect(image, provenance=provenance)
|
||||
``force`` bypasses detection for the mark's usual footprint).
|
||||
|
||||
``detection`` lets a caller that has ALREADY detected this mark at this trust
|
||||
level hand the result in rather than pay for the scan twice -- the explicit
|
||||
``visible --mark <name>`` path detects once to report the confidence and would
|
||||
otherwise re-detect here."""
|
||||
det = detection if detection is not None else self.detect(image, provenance=provenance)
|
||||
if not (det.detected or force):
|
||||
return Localization(det.detected, det.confidence, det.region, None)
|
||||
# Pass the (provenance-aware) detection to the mask builder so it does NOT
|
||||
@@ -272,6 +315,7 @@ class KnownMark:
|
||||
backend: Backend = "auto",
|
||||
provenance: bool = False,
|
||||
force: bool = False,
|
||||
detection: MarkDetection | None = None,
|
||||
) -> tuple[NDArray[Any], Region | None]:
|
||||
"""Remove this mark by localize -> fill; returns ``(result, region)`` where
|
||||
``region`` is the removed mark's bbox, or None if nothing was removed.
|
||||
@@ -282,7 +326,7 @@ class KnownMark:
|
||||
usual footprint even without a positive detection (the ``--no-detect`` path).
|
||||
NB: the CLI does NOT use ``region`` to clear alpha on save -- that zeroing
|
||||
caused the issue-#30 white box."""
|
||||
loc = self.localize(image, provenance=provenance, force=force)
|
||||
loc = self.localize(image, provenance=provenance, force=force, detection=detection)
|
||||
if loc.mask is None or not loc.mask.any():
|
||||
return image.copy(), None
|
||||
return fill(image, loc.mask, backend=backend), (loc.region if loc.detected else None)
|
||||
@@ -349,55 +393,31 @@ _GEMINI_PROVENANCE_MIN_CONF = 0.42
|
||||
|
||||
_engines: dict[str, Any] = {}
|
||||
|
||||
# key -> (module basename, class name). Only the NAMES live here: ``import_module``
|
||||
# runs inside :func:`_engine` on first use, so importing this module -- and with it the
|
||||
# metadata-only ``identify`` / ``visible_provenance`` path -- never pulls cv2 through
|
||||
# an engine. Same lazy-import shape as ``_text_mark_engine._rival_config`` and ``fill``.
|
||||
_ENGINE_CLASS: dict[str, tuple[str, str]] = {
|
||||
"gemini": ("gemini_engine", "GeminiEngine"),
|
||||
"doubao": ("doubao_engine", "DoubaoEngine"),
|
||||
"jimeng": ("jimeng_engine", "JimengEngine"),
|
||||
"qwen": ("qwen_engine", "QwenEngine"),
|
||||
"kling": ("kling_engine", "KlingEngine"),
|
||||
"yuanbao": ("yuanbao_engine", "YuanbaoEngine"),
|
||||
"samsung": ("samsung_engine", "SamsungEngine"),
|
||||
"jimeng_pill": ("pill_engine", "PillEngine"),
|
||||
"runninghub": ("runninghub_engine", "RunningHubEngine"),
|
||||
"baidu": ("baidu_engine", "BaiduEngine"),
|
||||
"liblib": ("liblib_engine", "LibLibEngine"),
|
||||
}
|
||||
|
||||
|
||||
def _engine(key: str) -> Any:
|
||||
if key not in _engines:
|
||||
if key == "gemini":
|
||||
from remove_ai_watermarks.gemini_engine import GeminiEngine
|
||||
from importlib import import_module
|
||||
|
||||
_engines[key] = GeminiEngine()
|
||||
elif key == "doubao":
|
||||
from remove_ai_watermarks.doubao_engine import DoubaoEngine
|
||||
|
||||
_engines[key] = DoubaoEngine()
|
||||
elif key == "jimeng":
|
||||
from remove_ai_watermarks.jimeng_engine import JimengEngine
|
||||
|
||||
_engines[key] = JimengEngine()
|
||||
elif key == "qwen":
|
||||
from remove_ai_watermarks.qwen_engine import QwenEngine
|
||||
|
||||
_engines[key] = QwenEngine()
|
||||
elif key == "kling":
|
||||
from remove_ai_watermarks.kling_engine import KlingEngine
|
||||
|
||||
_engines[key] = KlingEngine()
|
||||
elif key == "yuanbao":
|
||||
from remove_ai_watermarks.yuanbao_engine import YuanbaoEngine
|
||||
|
||||
_engines[key] = YuanbaoEngine()
|
||||
elif key == "samsung":
|
||||
from remove_ai_watermarks.samsung_engine import SamsungEngine
|
||||
|
||||
_engines[key] = SamsungEngine()
|
||||
elif key == "jimeng_pill":
|
||||
from remove_ai_watermarks.pill_engine import PillEngine
|
||||
|
||||
_engines[key] = PillEngine()
|
||||
elif key == "runninghub":
|
||||
from remove_ai_watermarks.runninghub_engine import RunningHubEngine
|
||||
|
||||
_engines[key] = RunningHubEngine()
|
||||
elif key == "baidu":
|
||||
from remove_ai_watermarks.baidu_engine import BaiduEngine
|
||||
|
||||
_engines[key] = BaiduEngine()
|
||||
elif key == "liblib":
|
||||
from remove_ai_watermarks.liblib_engine import LibLibEngine
|
||||
|
||||
_engines[key] = LibLibEngine()
|
||||
else: # pragma: no cover - guarded by the registry keys
|
||||
raise KeyError(key)
|
||||
module_name, class_name = _ENGINE_CLASS[key] # KeyError(key) for an unknown key
|
||||
_engines[key] = getattr(import_module(f"remove_ai_watermarks.{module_name}"), class_name)()
|
||||
return _engines[key]
|
||||
|
||||
|
||||
@@ -462,13 +482,21 @@ def fill(image: NDArray[Any], mask: NDArray[Any], *, backend: Backend = "auto")
|
||||
# detector on the memory-tight identify host), so detection never builds a mask.
|
||||
|
||||
|
||||
def _gemini_detect(image: NDArray[Any], *, provenance: bool = False) -> MarkDetection:
|
||||
d = _engine("gemini").detect_watermark(image, trust_provenance=provenance)
|
||||
def _gemini_wrap(d: Any, *, provenance: bool) -> MarkDetection:
|
||||
gate = _GEMINI_PROVENANCE_MIN_CONF if provenance else _GEMINI_AUTO_MIN_CONF
|
||||
detected = bool(d.detected) and d.confidence >= gate
|
||||
return MarkDetection("gemini", "Google Gemini sparkle", "bottom-right", detected, d.confidence, d.region)
|
||||
|
||||
|
||||
def _gemini_detect(image: NDArray[Any], *, provenance: bool = False) -> MarkDetection:
|
||||
return _gemini_wrap(_engine("gemini").detect_watermark(image, trust_provenance=provenance), provenance=provenance)
|
||||
|
||||
|
||||
def _gemini_detect_both(image: NDArray[Any]) -> tuple[MarkDetection, MarkDetection]:
|
||||
strict, relaxed = _engine("gemini").detect_watermark_both(image)
|
||||
return _gemini_wrap(strict, provenance=False), _gemini_wrap(relaxed, provenance=True)
|
||||
|
||||
|
||||
def _gemini_mask(
|
||||
image: NDArray[Any], *, force: bool = False, detection: MarkDetection | None = None
|
||||
) -> NDArray[Any] | None:
|
||||
@@ -487,26 +515,73 @@ def _gemini_mask(
|
||||
def _text_mark_detect(key: str, label: str, location: str) -> Callable[..., MarkDetection]:
|
||||
def detect(image: NDArray[Any], *, provenance: bool = False) -> MarkDetection:
|
||||
d = _engine(key).detect(image, provenance=provenance)
|
||||
return MarkDetection(key, label, location, d.detected, d.confidence, d.region)
|
||||
return MarkDetection(key, label, location, d.detected, d.confidence, d.region, engine_detection=d)
|
||||
|
||||
return detect
|
||||
|
||||
|
||||
def _text_mark_detect_both(key: str, label: str, location: str) -> Callable[..., tuple[MarkDetection, MarkDetection]]:
|
||||
def detect_both(image: NDArray[Any]) -> tuple[MarkDetection, MarkDetection]:
|
||||
strict, relaxed = _engine(key).detect_both(image)
|
||||
return (
|
||||
MarkDetection(
|
||||
key, label, location, strict.detected, strict.confidence, strict.region, engine_detection=strict
|
||||
),
|
||||
MarkDetection(
|
||||
key, label, location, relaxed.detected, relaxed.confidence, relaxed.region, engine_detection=relaxed
|
||||
),
|
||||
)
|
||||
|
||||
return detect_both
|
||||
|
||||
|
||||
def _text_mark_mask(key: str) -> Callable[..., NDArray[Any] | None]:
|
||||
def mask(
|
||||
image: NDArray[Any], *, force: bool = False, detection: MarkDetection | None = None
|
||||
) -> NDArray[Any] | None:
|
||||
# Text masks rebuild the glyph blob template-free (no trust gate to re-apply), so
|
||||
# the detection is not needed here; accepted for the uniform _mask signature.
|
||||
del detection
|
||||
return _engine(key).footprint_mask(image, force=force)
|
||||
# Thread the engine's OWN detection into the mask builder: the footprint is
|
||||
# bounded by the ladder sweep the detector already ran, so re-detecting here
|
||||
# repeated locate + extract_mask + an identical sweep. footprint_mask still
|
||||
# re-detects when nothing is threaded (a direct or --no-detect caller) and when
|
||||
# the threaded detection was taken at a relaxed trust level.
|
||||
return _engine(key).footprint_mask(
|
||||
image, force=force, detection=detection.engine_detection if detection is not None else None
|
||||
)
|
||||
|
||||
return mask
|
||||
|
||||
|
||||
def _text_mark(key: str, label: str, location: str) -> KnownMark:
|
||||
"""Build a text-mark registry row from its shared detector and mask adapters."""
|
||||
return KnownMark(key, label, location, True, _text_mark_detect(key, label, location), _text_mark_mask(key))
|
||||
def _text_mark(
|
||||
key: str,
|
||||
label: str,
|
||||
location: str,
|
||||
*,
|
||||
platform: str,
|
||||
product: str | None = None,
|
||||
label_regime: str | None = "tc260",
|
||||
provenance_signals: tuple[str, ...] = ("aigc",),
|
||||
tc260_producer_codes: tuple[str, ...] = (),
|
||||
) -> KnownMark:
|
||||
"""Build a text-mark registry row from its shared detector and mask adapters.
|
||||
|
||||
``product`` defaults to the key (one mark, one product); pass it only when two
|
||||
marks share a product. ``label_regime`` and ``provenance_signals`` default to the
|
||||
China-AIGC label because every text mark registered so far except Samsung uses it.
|
||||
"""
|
||||
return KnownMark(
|
||||
key,
|
||||
label,
|
||||
location,
|
||||
True,
|
||||
product or key,
|
||||
label_regime,
|
||||
platform,
|
||||
_text_mark_detect(key, label, location),
|
||||
_text_mark_mask(key),
|
||||
provenance_signals=provenance_signals,
|
||||
tc260_producer_codes=tc260_producer_codes,
|
||||
_detect_both=_text_mark_detect_both(key, label, location),
|
||||
)
|
||||
|
||||
|
||||
# ── Capture-less mark: the Jimeng-basic "AI生成" pill (top-left) ──
|
||||
@@ -518,6 +593,13 @@ def _pill_detect(image: NDArray[Any], *, provenance: bool = False) -> MarkDetect
|
||||
return MarkDetection("jimeng_pill", "Jimeng AI生成 pill", "top-left", d.detected, d.confidence, d.region)
|
||||
|
||||
|
||||
def _pill_detect_both(image: NDArray[Any]) -> tuple[MarkDetection, MarkDetection]:
|
||||
# The pill detector is provenance-independent (`_pill_detect` discards the flag), so
|
||||
# one call answers both levels. MarkDetection is frozen, so sharing it is safe.
|
||||
d = _pill_detect(image)
|
||||
return d, d
|
||||
|
||||
|
||||
def _pill_mask(
|
||||
image: NDArray[Any], *, force: bool = False, detection: MarkDetection | None = None
|
||||
) -> NDArray[Any] | None:
|
||||
@@ -534,19 +616,110 @@ def _pill_features(image: NDArray[Any]) -> dict[str, float]:
|
||||
|
||||
|
||||
_REGISTRY: tuple[KnownMark, ...] = (
|
||||
KnownMark("gemini", "Google Gemini sparkle", "bottom-right", True, _gemini_detect, _gemini_mask),
|
||||
_text_mark("doubao", "Doubao 豆包AI生成 text", "bottom-right"),
|
||||
_text_mark("jimeng", "Jimeng 即梦AI wordmark", "bottom-right"),
|
||||
_text_mark("qwen", "Qwen 千问AI生成 text", "bottom-right"),
|
||||
_text_mark("kling", "Kling 可灵AI 3.0 text", "bottom-right"),
|
||||
_text_mark("yuanbao", "Tencent Yuanbao 元宝 / AI生成 mark", "bottom-right"),
|
||||
_text_mark("samsung", "Samsung Galaxy AI text", "bottom-left"),
|
||||
_text_mark("runninghub", "RunningHub AI生成 text", "top-left"),
|
||||
_text_mark("baidu", "Baidu 百度 AI生成 text", "bottom-right"),
|
||||
_text_mark("liblib", "LibLibAI wordmark", "bottom-center"),
|
||||
KnownMark("jimeng_pill", "Jimeng AI生成 pill", "top-left", True, _pill_detect, _pill_mask, _pill_features),
|
||||
# Gemini is a Google C2PA/SynthID product, not a China-AIGC labeller: label_regime
|
||||
# is None so it can never act as a TC260 sibling in _keep_pill.
|
||||
KnownMark(
|
||||
"gemini",
|
||||
"Google Gemini sparkle",
|
||||
"bottom-right",
|
||||
True,
|
||||
"gemini",
|
||||
None,
|
||||
# No platform sentence: the sparkle has its own higher-confidence
|
||||
# `_visible_sparkle` path in identify, which names the platform itself.
|
||||
None,
|
||||
_gemini_detect,
|
||||
_gemini_mask,
|
||||
provenance_platform_tokens=("google", "gemini"),
|
||||
_detect_both=_gemini_detect_both,
|
||||
),
|
||||
_text_mark(
|
||||
"doubao",
|
||||
"Doubao 豆包AI生成 text",
|
||||
"bottom-right",
|
||||
platform="ByteDance Doubao (visible 豆包AI生成 mark detected)",
|
||||
tc260_producer_codes=("91110102MACQD9K640", "doubao"),
|
||||
),
|
||||
_text_mark(
|
||||
"jimeng",
|
||||
"Jimeng 即梦AI wordmark",
|
||||
"bottom-right",
|
||||
platform="ByteDance Jimeng / Dreamina (visible 即梦AI mark detected)",
|
||||
tc260_producer_codes=("9144030008867405X2",),
|
||||
),
|
||||
_text_mark(
|
||||
"qwen",
|
||||
"Qwen 千问AI生成 text",
|
||||
"bottom-right",
|
||||
platform="Alibaba Tongyi Qianwen (visible 千问AI生成 mark detected)",
|
||||
tc260_producer_codes=("91440101MA9Y9T4H7A",),
|
||||
),
|
||||
_text_mark(
|
||||
"kling",
|
||||
"Kling 可灵AI 3.0 text",
|
||||
"bottom-right",
|
||||
platform="Kuaishou Kling (visible 可灵AI 3.0 mark detected)",
|
||||
tc260_producer_codes=("91110108335469089C",),
|
||||
),
|
||||
_text_mark(
|
||||
"yuanbao",
|
||||
"Tencent Yuanbao 元宝 / AI生成 mark",
|
||||
"bottom-right",
|
||||
platform="Tencent Yuanbao (visible 元宝 / AI生成 mark detected)",
|
||||
tc260_producer_codes=("91440300708461136T",),
|
||||
),
|
||||
# Samsung Galaxy AI is a device editing marker (samsung_genai), not a TC260 label.
|
||||
_text_mark(
|
||||
"samsung",
|
||||
"Samsung Galaxy AI text",
|
||||
"bottom-left",
|
||||
label_regime=None,
|
||||
provenance_signals=("samsung_genai",),
|
||||
platform="Samsung Galaxy AI (visible 'Contenuti generati dall'AI' mark detected)",
|
||||
),
|
||||
_text_mark(
|
||||
"runninghub",
|
||||
"RunningHub AI生成 text",
|
||||
"top-left",
|
||||
platform="RunningHub (visible RunningHub AI生成 mark detected)",
|
||||
tc260_producer_codes=("91340100MAEB4N8H76", "RunningHub"),
|
||||
),
|
||||
_text_mark(
|
||||
"baidu",
|
||||
"Baidu 百度 AI生成 text",
|
||||
"bottom-right",
|
||||
platform="Baidu (visible 百度 AI生成 mark detected)",
|
||||
tc260_producer_codes=("91110000802100433B",),
|
||||
),
|
||||
_text_mark(
|
||||
"liblib",
|
||||
"LibLibAI wordmark",
|
||||
"bottom-center",
|
||||
platform="LibLibAI (visible LibLibAI mark detected)",
|
||||
tc260_producer_codes=("91110105MACJ6K1C8A",),
|
||||
),
|
||||
# Same product as the Jimeng wordmark -- the one pair that cross-relaxes.
|
||||
KnownMark(
|
||||
"jimeng_pill",
|
||||
"Jimeng AI生成 pill",
|
||||
"top-left",
|
||||
True,
|
||||
"jimeng",
|
||||
"tc260",
|
||||
# The capture-less pill is too weak a detector to attribute a platform on its
|
||||
# own; the Jimeng wordmark is what names ByteDance.
|
||||
None,
|
||||
_pill_detect,
|
||||
_pill_mask,
|
||||
_pill_features,
|
||||
_detect_both=_pill_detect_both,
|
||||
),
|
||||
)
|
||||
|
||||
# Product family per mark, derived from the registry rows so registering a mark is one
|
||||
# edit. See KnownMark.product for why Doubao and Jimeng must not cross-relax.
|
||||
_PRODUCT_OF: dict[str, str] = {m.key: m.product for m in _REGISTRY}
|
||||
|
||||
|
||||
def known_marks() -> tuple[KnownMark, ...]:
|
||||
"""All registered known visible watermarks."""
|
||||
@@ -606,6 +779,30 @@ def resolve_trust(
|
||||
return "confirmed" if confirmed else "strict"
|
||||
|
||||
|
||||
def tc260_producer_vendors() -> dict[str, str]:
|
||||
"""TC260 ``ContentProducer`` identity -> the mark key whose vendor signs with it.
|
||||
|
||||
Derived from the registry rows, so registering a TC260 mark and its producer codes
|
||||
is one edit. A mark registered without codes falls through to
|
||||
:data:`TC260_FALLBACK_VENDORS`, which relaxes ByteDance's pair -- a silent wrong
|
||||
answer on an image carrying the new mark, which is why the codes belong on the row
|
||||
next to ``label_regime`` rather than in a table someone must remember to update.
|
||||
"""
|
||||
return {code: mark.key for mark in _REGISTRY for code in mark.tc260_producer_codes}
|
||||
|
||||
|
||||
def _pill_suppressors() -> set[str]:
|
||||
"""Marks whose detection vetoes the capture-less pill: same label regime as the
|
||||
pill, different product. Derived so a newly registered TC260 mark cannot be
|
||||
forgotten here -- which is exactly how LibLibAI ended up missing."""
|
||||
pill = get_mark("jimeng_pill")
|
||||
return {
|
||||
m.key
|
||||
for m in _REGISTRY
|
||||
if m.label_regime is not None and m.label_regime == pill.label_regime and m.product != pill.product
|
||||
}
|
||||
|
||||
|
||||
def _keep_pill(keys: set[str], *, provenance: frozenset[str], footprint_flat: bool) -> bool:
|
||||
"""Whether to auto-remove the capture-less 'AI生成' pill given the fired marks.
|
||||
|
||||
@@ -620,17 +817,18 @@ def _keep_pill(keys: set[str], *, provenance: frozenset[str], footprint_flat: bo
|
||||
so real flat-scene pills (and harmless flat false fires) are cleaned while the
|
||||
damaging textured false fires are left untouched.
|
||||
A Doubao image is TC260 too but is not Jimeng-basic, so the pill never rides on a
|
||||
Doubao detection; Qwen, Kling, Yuanbao, RunningHub, and Baidu detections likewise
|
||||
name other products and suppress the pill.
|
||||
No confirmation at all -> never remove (blocks false fires on non-Jimeng content)."""
|
||||
if (
|
||||
"doubao" in keys
|
||||
or "qwen" in keys
|
||||
or "kling" in keys
|
||||
or "yuanbao" in keys
|
||||
or "runninghub" in keys
|
||||
or "baidu" in keys
|
||||
):
|
||||
Doubao detection; every other TC260 product's mark likewise names a different
|
||||
product and suppresses the pill.
|
||||
No confirmation at all -> never remove (blocks false fires on non-Jimeng content).
|
||||
|
||||
The suppressor set is DERIVED from the registry (same label regime, different
|
||||
product), not hand-listed. The hand-written list had drifted: LibLibAI was
|
||||
registered alongside RunningHub and Baidu but never added to it, so a confident
|
||||
LibLibAI detection did not veto the pill the way its two siblings did. Marks
|
||||
outside the TC260 regime (Gemini, Samsung) are deliberately NOT suppressors --
|
||||
neither can put ``"jimeng"`` into ``provenance``, so neither can enable the arm
|
||||
they would be vetoing."""
|
||||
if _pill_suppressors() & keys:
|
||||
return False
|
||||
if "jimeng" in keys:
|
||||
return True
|
||||
@@ -647,15 +845,17 @@ def _build_candidates(image: NDArray[Any]) -> list[Candidate]:
|
||||
Each mark is detected at the strict AND the relaxed (``provenance=True``) level so
|
||||
:func:`decide` can pick per mark without re-running detection; a relaxed gate is
|
||||
monotonically more permissive, so this reproduces the old strict-then-relax pass
|
||||
exactly. The loop is uniform -- it knows nothing about any specific mark: each mark
|
||||
reports its own gate features via :meth:`KnownMark.features` (computed only when the
|
||||
mark is detected, so a clean image pays nothing extra)."""
|
||||
exactly. Both levels come from ONE scan per mark (:meth:`KnownMark.detect_both`):
|
||||
the trust level moves a threshold, never the measurement, so running the detector
|
||||
twice was doing the expensive half of the work for a second time. The loop is
|
||||
uniform -- it knows nothing about any specific mark: each mark reports its own gate
|
||||
features via :meth:`KnownMark.features` (computed only when the mark is detected, so
|
||||
a clean image pays nothing extra)."""
|
||||
cands: list[Candidate] = []
|
||||
for m in _REGISTRY:
|
||||
if not m.in_auto:
|
||||
continue
|
||||
strict = m.detect(image, provenance=False)
|
||||
relaxed = m.detect(image, provenance=True)
|
||||
strict, relaxed = m.detect_both(image)
|
||||
feats = m.features(image) if (strict.detected or relaxed.detected) else {}
|
||||
cands.append(Candidate(m.key, m.label, strict.detected, relaxed.detected, feats))
|
||||
return cands
|
||||
|
||||
@@ -26,13 +26,16 @@ import logging
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from remove_ai_watermarks import _text_mark_engine
|
||||
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
|
||||
from remove_ai_watermarks._text_mark_engine import (
|
||||
TextMarkConfig,
|
||||
TextMarkDetection,
|
||||
TextMarkEngine,
|
||||
TextMarkScan,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
WM_WIDTH_FRAC = 0.20
|
||||
@@ -75,24 +78,12 @@ _CONFIG = TextMarkConfig(
|
||||
provenance_ncc_factor=1.0,
|
||||
)
|
||||
|
||||
YuanbaoDetection = TextMarkDetection
|
||||
|
||||
|
||||
def _alpha_template() -> NDArray[Any] | None:
|
||||
"""The bundled Yuanbao alpha template (float [0,1]), or None."""
|
||||
return _text_mark_engine.load_alpha_template(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _glyph_silhouette() -> NDArray[Any] | None:
|
||||
"""Binary two-line Yuanbao silhouette (255 = glyph), or None."""
|
||||
return _text_mark_engine.glyph_silhouette(_CONFIG.asset_name)
|
||||
|
||||
|
||||
def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
|
||||
"""TM_CCOEFF_NORMED of the Yuanbao silhouette against ``box_mask``."""
|
||||
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
|
||||
|
||||
|
||||
class YuanbaoEngine(TextMarkEngine):
|
||||
"""Detect and localize the bottom-right Yuanbao mark."""
|
||||
|
||||
@@ -102,37 +93,28 @@ class YuanbaoEngine(TextMarkEngine):
|
||||
def __init__(self) -> None:
|
||||
super().__init__(_CONFIG)
|
||||
|
||||
def detect(self, image: NDArray[Any] | None, *, provenance: bool = False) -> TextMarkDetection:
|
||||
if image is None or not image.size:
|
||||
return TextMarkDetection()
|
||||
detection = super().detect(image, provenance=provenance)
|
||||
if not detection.detected:
|
||||
return detection
|
||||
location = self.locate(image)
|
||||
_, box = self._contrast_best(image, location)
|
||||
def _post_gate(self, det: TextMarkDetection, scan: TextMarkScan) -> TextMarkDetection:
|
||||
"""Demote a match that does not hug the bottom-right corner.
|
||||
|
||||
A shared post-gate rather than a ``detect`` override, so the single-pass
|
||||
perception path (``detect_both``) cannot skip it.
|
||||
"""
|
||||
if not det.detected or scan.loc is None:
|
||||
return det
|
||||
box = det.match_box # the sweep the scan already ran on this same loc
|
||||
if box is None:
|
||||
detection.detected = False
|
||||
return detection
|
||||
h, w = image.shape[:2]
|
||||
det.detected = False
|
||||
return det
|
||||
h, w = scan.frame
|
||||
base = min(h, w)
|
||||
right = (w - (location.x + box[2] + 1)) / base
|
||||
bottom = (h - (location.y + box[3] + 1)) / base
|
||||
right = (w - (scan.loc.x + box[2] + 1)) / base
|
||||
bottom = (h - (scan.loc.y + box[3] + 1)) / base
|
||||
if not (0 <= right <= self._ANCHOR_MAX_RIGHT and 0 <= bottom <= self._ANCHOR_MAX_BOTTOM):
|
||||
logger.debug(
|
||||
"Yuanbao detect: score %.3f but match off-anchor (right=%.3f bottom=%.3f); demoting.",
|
||||
detection.confidence,
|
||||
det.confidence,
|
||||
right,
|
||||
bottom,
|
||||
)
|
||||
detection.detected = False
|
||||
return detection
|
||||
|
||||
|
||||
def load_image_bgr(path: str | Path) -> NDArray[Any]:
|
||||
"""Read an image as a BGR ndarray."""
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
image = image_io.imread(path)
|
||||
if image is None:
|
||||
raise FileNotFoundError(f"Failed to read image: {path}")
|
||||
return image
|
||||
det.detected = False
|
||||
return det
|
||||
|
||||
+313
-3
@@ -91,10 +91,59 @@ class TestNoOpPreservesOriginal:
|
||||
|
||||
class TestVisibleProvenance:
|
||||
@pytest.mark.skipif(not DOUBAO.exists(), reason="doubao sample not present")
|
||||
def test_doubao_tc260_maps_to_bytedance(self):
|
||||
def test_doubao_tc260_maps_to_the_producer_it_names(self):
|
||||
"""The TC260 producer identifies the vendor, so only Doubao is relaxed.
|
||||
|
||||
This used to relax Doubao AND Jimeng on every China-AIGC image, because the
|
||||
label alone does not say which vendor made it. Its ``ContentProducer`` does.
|
||||
"""
|
||||
prov = raiw.visible_provenance(DOUBAO)
|
||||
# TC260 label -> ByteDance family (both doubao and jimeng)
|
||||
assert {"doubao", "jimeng"} <= prov
|
||||
assert "doubao" in prov
|
||||
assert "jimeng" not in prov
|
||||
|
||||
def test_unmapped_tc260_producer_falls_back_to_the_bytedance_pair(self, monkeypatch, tmp_path):
|
||||
"""An unrecognized producer must not lose the relaxation entirely: the label is
|
||||
still evidence that some China-AIGC vendor made the image."""
|
||||
from types import SimpleNamespace
|
||||
|
||||
from remove_ai_watermarks import identify, metadata
|
||||
|
||||
monkeypatch.setattr(
|
||||
identify,
|
||||
"identify",
|
||||
lambda *a, **k: SimpleNamespace(platform=None, signals=[SimpleNamespace(name="aigc")]),
|
||||
)
|
||||
monkeypatch.setattr(metadata, "aigc_label", lambda _p: {"ContentProducer": "0011999999999999999999999"})
|
||||
assert raiw.visible_provenance(tmp_path / "x.png") == frozenset({"doubao", "jimeng"})
|
||||
|
||||
def test_known_tc260_producer_names_a_single_vendor(self, monkeypatch, tmp_path):
|
||||
from types import SimpleNamespace
|
||||
|
||||
from remove_ai_watermarks import identify, metadata
|
||||
|
||||
monkeypatch.setattr(
|
||||
identify,
|
||||
"identify",
|
||||
lambda *a, **k: SimpleNamespace(platform=None, signals=[SimpleNamespace(name="aigc")]),
|
||||
)
|
||||
# 001 + 1 + USCC(18) + 5-digit product suffix, the Qwen entity.
|
||||
monkeypatch.setattr(metadata, "aigc_label", lambda _p: {"ContentProducer": "001191440101MA9Y9T4H7A00001"})
|
||||
assert raiw.visible_provenance(tmp_path / "x.png") == frozenset({"qwen"})
|
||||
|
||||
def test_every_mapped_producer_names_a_registered_mark(self):
|
||||
"""The table drives the arbiter's provenance set, so a typo'd key would relax
|
||||
nothing and fail silently. Checked here, not at import: importing the registry
|
||||
from _internal.constants would drag it into every metadata-only path."""
|
||||
from remove_ai_watermarks._internal.constants import TC260_FALLBACK_VENDORS
|
||||
from remove_ai_watermarks.watermark_registry import known_marks, mark_keys, tc260_producer_vendors
|
||||
|
||||
keys = set(mark_keys())
|
||||
assert set(tc260_producer_vendors().values()) <= keys
|
||||
assert keys >= TC260_FALLBACK_VENDORS
|
||||
# Every TC260 mark should name its producer, or it silently falls back to the
|
||||
# ByteDance pair on an image carrying that mark.
|
||||
unmapped = {m.key for m in known_marks() if m.label_regime == "tc260" and not m.tc260_producer_codes}
|
||||
assert unmapped == {"jimeng_pill"}, f"TC260 marks with no producer code: {unmapped}"
|
||||
|
||||
@pytest.mark.skipif(not CHATGPT.exists(), reason="chatgpt sample not present")
|
||||
def test_openai_image_has_no_visible_vendor(self):
|
||||
@@ -144,3 +193,264 @@ class TestRemoveVisibleOutputPath:
|
||||
out = tmp_path / "sub" / "out.png"
|
||||
raiw.remove_visible(str(src), str(out), backend="cv2")
|
||||
assert out.exists()
|
||||
|
||||
|
||||
class TestRemoveAllLibrary:
|
||||
"""The three-stage pipeline is a library function, not CLI-only.
|
||||
|
||||
It used to live only in ``cli.py``, written twice (once for ``all``, once for
|
||||
``batch``) with divergent behavior, so no library caller could reach it.
|
||||
"""
|
||||
|
||||
def _patched_engine(self, monkeypatch, calls: list):
|
||||
from remove_ai_watermarks import invisible_engine
|
||||
|
||||
class FakeEngine:
|
||||
def remove_watermark(self, *args, **kwargs):
|
||||
calls.append(kwargs.get("image_path") or args[0])
|
||||
|
||||
monkeypatch.setattr(invisible_engine, "is_available", lambda: True)
|
||||
return FakeEngine()
|
||||
|
||||
@pytest.mark.skipif(not DOUBAO.exists(), reason="doubao sample not present")
|
||||
def test_runs_all_three_stages_and_reports_each(self, monkeypatch, tmp_path):
|
||||
from remove_ai_watermarks import api
|
||||
|
||||
calls: list = []
|
||||
engine = self._patched_engine(monkeypatch, calls)
|
||||
monkeypatch.setattr(api._SourceEvidence, "has_invisible_target", lambda _self: True)
|
||||
out = tmp_path / "clean.png"
|
||||
events: list[tuple[str, str]] = []
|
||||
|
||||
result = api.remove_all(DOUBAO, out, backend="cv2", engine=engine, progress=lambda s, d: events.append((s, d)))
|
||||
|
||||
assert out.exists()
|
||||
assert result.invisible == "removed"
|
||||
assert result.visible_label is not None # the Doubao mark fired
|
||||
assert calls, "the invisible engine was never invoked"
|
||||
# Progress is (stage, stable-token), never prose the caller has to parse back.
|
||||
assert any(stage == "visible" for stage, _ in events)
|
||||
assert ("invisible", "removed") in events
|
||||
assert ("metadata", "stripped") in events
|
||||
|
||||
@pytest.mark.skipif(not DOUBAO.exists(), reason="doubao sample not present")
|
||||
def test_no_signal_skips_the_scrub_but_still_writes(self, monkeypatch, tmp_path):
|
||||
from remove_ai_watermarks import api
|
||||
|
||||
calls: list = []
|
||||
engine = self._patched_engine(monkeypatch, calls)
|
||||
monkeypatch.setattr(api._SourceEvidence, "has_invisible_target", lambda _self: False)
|
||||
out = tmp_path / "clean.png"
|
||||
|
||||
result = api.remove_all(DOUBAO, out, backend="cv2", engine=engine)
|
||||
|
||||
assert result.invisible == "no-signal"
|
||||
assert not calls
|
||||
assert out.exists() # a deliberate skip is still a successful run
|
||||
|
||||
@pytest.mark.skipif(not DOUBAO.exists(), reason="doubao sample not present")
|
||||
def test_missing_gpu_extra_is_reported_not_raised(self, monkeypatch, tmp_path):
|
||||
from remove_ai_watermarks import api, invisible_engine
|
||||
|
||||
monkeypatch.setattr(invisible_engine, "is_available", lambda: False)
|
||||
out = tmp_path / "clean.png"
|
||||
|
||||
result = api.remove_all(DOUBAO, out, backend="cv2")
|
||||
|
||||
assert result.invisible == "unavailable"
|
||||
assert out.exists() # it LOOKS processed -- which is why the caller must warn
|
||||
|
||||
@pytest.mark.skipif(not DOUBAO.exists(), reason="doubao sample not present")
|
||||
def test_incomplete_strip_leaves_no_output_file(self, monkeypatch, tmp_path):
|
||||
"""The contract the CLI depends on: an AI-readable output plus a non-zero exit
|
||||
is worse than no output at all, so the raise happens BEFORE the final write."""
|
||||
from remove_ai_watermarks import api, invisible_engine, metadata
|
||||
|
||||
monkeypatch.setattr(invisible_engine, "is_available", lambda: False)
|
||||
monkeypatch.setattr(metadata, "strip_and_verify", lambda src, dst: (dst, {"c2pa"}))
|
||||
out = tmp_path / "clean.png"
|
||||
|
||||
with pytest.raises(api.MetadataStripIncomplete, match="c2pa"):
|
||||
api.remove_all(DOUBAO, out, backend="cv2")
|
||||
assert not out.exists()
|
||||
|
||||
def test_unreadable_source_raises_valueerror(self, tmp_path):
|
||||
from remove_ai_watermarks import api
|
||||
|
||||
with pytest.raises(ValueError, match="Could not read image"):
|
||||
api.remove_all(tmp_path / "nope.png", tmp_path / "out.png")
|
||||
|
||||
@pytest.mark.skipif(not DOUBAO.exists(), reason="doubao sample not present")
|
||||
def test_stages_through_the_system_temp_dir_not_the_output_dir(self, monkeypatch, tmp_path):
|
||||
"""Staging next to the output would defeat the point: the user must not see a
|
||||
partial file there during a long model download."""
|
||||
from remove_ai_watermarks import api, invisible_engine
|
||||
|
||||
monkeypatch.setattr(invisible_engine, "is_available", lambda: False)
|
||||
out_dir = tmp_path / "out"
|
||||
out_dir.mkdir()
|
||||
seen: list[set[str]] = []
|
||||
|
||||
real = api.remove_all
|
||||
|
||||
def spy(*args, **kwargs):
|
||||
result = real(*args, **kwargs)
|
||||
seen.append({p.name for p in out_dir.iterdir()})
|
||||
return result
|
||||
|
||||
spy(DOUBAO, out_dir / "clean.png", backend="cv2")
|
||||
assert seen == [{"clean.png"}], "an intermediate was left in the output directory"
|
||||
|
||||
|
||||
class TestRemoveBatchLibrary:
|
||||
@pytest.mark.skipif(not DOUBAO.exists(), reason="doubao sample not present")
|
||||
def test_visible_mode_writes_every_image(self, tmp_path):
|
||||
import shutil
|
||||
|
||||
from remove_ai_watermarks import api
|
||||
|
||||
src = tmp_path / "in"
|
||||
src.mkdir()
|
||||
for i in range(3):
|
||||
shutil.copyfile(DOUBAO, src / f"img{i}.png")
|
||||
out = tmp_path / "out"
|
||||
|
||||
summary = api.remove_batch(src, out, mode="visible", backend="cv2")
|
||||
|
||||
assert summary.processed == 3
|
||||
assert summary.failed == 0
|
||||
assert sorted(p.name for p in out.iterdir()) == ["img0.png", "img1.png", "img2.png"]
|
||||
|
||||
def test_one_bad_file_does_not_abandon_the_rest(self, tmp_path):
|
||||
from remove_ai_watermarks import api
|
||||
|
||||
src = tmp_path / "in"
|
||||
src.mkdir()
|
||||
(src / "broken.png").write_bytes(b"not a png at all")
|
||||
image = np.full((64, 64, 3), 120, np.uint8)
|
||||
raiw.remove_visible(image) # sanity: the registry is importable here
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
image_io.imwrite(src / "good.png", image)
|
||||
out = tmp_path / "out"
|
||||
|
||||
summary = api.remove_batch(src, out, mode="visible", backend="cv2")
|
||||
|
||||
assert summary.processed == 1
|
||||
assert summary.failed == 1
|
||||
assert [p.name for p, _ in summary.errors] == ["broken.png"]
|
||||
assert (out / "good.png").exists()
|
||||
|
||||
def test_a_failed_write_is_counted_not_swallowed(self, tmp_path, monkeypatch):
|
||||
"""Tier E: a read-only output directory once produced zero files and exit 0."""
|
||||
from remove_ai_watermarks import api, image_io
|
||||
|
||||
src = tmp_path / "in"
|
||||
src.mkdir()
|
||||
image_io.imwrite(src / "a.png", np.full((64, 64, 3), 120, np.uint8))
|
||||
monkeypatch.setattr(image_io, "write_bgr_with_alpha", lambda *a, **k: False)
|
||||
|
||||
summary = api.remove_batch(src, tmp_path / "out", mode="visible", backend="cv2")
|
||||
|
||||
assert summary.processed == 0
|
||||
assert summary.failed == 1
|
||||
|
||||
|
||||
class TestSourceEvidenceHolder:
|
||||
"""One metadata extraction per source file, per call.
|
||||
|
||||
``remove_all`` asks the same file two provenance questions (which vendor is
|
||||
confirmed, and is there an invisible target); both start from the same extraction.
|
||||
"""
|
||||
|
||||
@pytest.mark.skipif(not DOUBAO.exists(), reason="doubao sample not present")
|
||||
def test_remove_all_extracts_evidence_once(self, monkeypatch, tmp_path):
|
||||
import shutil
|
||||
|
||||
from remove_ai_watermarks import api, identify, invisible_engine
|
||||
|
||||
class Fake:
|
||||
def remove_watermark(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
source = tmp_path / "in.png"
|
||||
shutil.copyfile(DOUBAO, source)
|
||||
calls: list[int] = []
|
||||
real = identify.extract_provenance_evidence
|
||||
monkeypatch.setattr(identify, "extract_provenance_evidence", lambda p: (calls.append(1), real(p))[1])
|
||||
monkeypatch.setattr(invisible_engine, "is_available", lambda: True)
|
||||
|
||||
api.remove_all(source, tmp_path / "out.png", backend="cv2", engine=Fake())
|
||||
assert len(calls) == 1
|
||||
|
||||
@pytest.mark.skipif(not DOUBAO.exists(), reason="doubao sample not present")
|
||||
def test_holder_agrees_with_the_standalone_functions(self):
|
||||
from remove_ai_watermarks import api, identify
|
||||
|
||||
holder = api._SourceEvidence(DOUBAO)
|
||||
assert holder.visible_provenance() == api.visible_provenance(DOUBAO)
|
||||
assert holder.has_invisible_target() == identify.has_invisible_target(DOUBAO)
|
||||
|
||||
def test_extraction_failure_fails_safe_in_both_directions(self, monkeypatch, tmp_path):
|
||||
"""No provenance means no relaxation; an unknown invisible target means SCRUB.
|
||||
Leaving a watermark on a paid removal is worse than over-regenerating."""
|
||||
from remove_ai_watermarks import api, identify
|
||||
|
||||
def boom(_path):
|
||||
raise OSError("extract exploded")
|
||||
|
||||
monkeypatch.setattr(identify, "extract_provenance_evidence", boom)
|
||||
holder = api._SourceEvidence(tmp_path / "x.png")
|
||||
assert holder.visible_provenance() == frozenset()
|
||||
assert holder.has_invisible_target() is True
|
||||
|
||||
@pytest.mark.skipif(not DOUBAO.exists(), reason="doubao sample not present")
|
||||
def test_a_verdict_failure_also_fails_safe(self, monkeypatch):
|
||||
"""The suppress must span the VERDICT and the mapping, not just the extraction:
|
||||
a raise here used to escape as a traceback where the old code returned empty."""
|
||||
from remove_ai_watermarks import api, identify
|
||||
|
||||
def boom(*args, **kwargs):
|
||||
raise RuntimeError("verdict exploded")
|
||||
|
||||
monkeypatch.setattr(identify, "identify_from_evidence", boom)
|
||||
holder = api._SourceEvidence(DOUBAO)
|
||||
assert holder.visible_provenance() == frozenset()
|
||||
assert holder.has_invisible_target() is True
|
||||
|
||||
|
||||
class TestBatchProgressIsStructured:
|
||||
"""`remove_batch` emits exactly one terminal event per image, in every mode.
|
||||
|
||||
Regression: the CLI advanced its progress bar by string-matching a stage line that
|
||||
only ``mode="all"`` ever emitted, so a `visible` or `metadata` batch sat at 0% for
|
||||
the whole run and jumped to 100% at the end.
|
||||
"""
|
||||
|
||||
def _run(self, tmp_path, mode: str) -> list[tuple[str, str, str]]:
|
||||
from remove_ai_watermarks import api, image_io
|
||||
|
||||
src = tmp_path / "in"
|
||||
src.mkdir()
|
||||
for i in range(3):
|
||||
image_io.imwrite(src / f"img{i}.png", np.full((64, 64, 3), 120, np.uint8))
|
||||
events: list[tuple[str, str, str]] = []
|
||||
api.remove_batch(
|
||||
src,
|
||||
tmp_path / "out",
|
||||
mode=mode, # type: ignore[arg-type]
|
||||
backend="cv2",
|
||||
progress=lambda p, stage, detail: events.append((p.name, stage, detail)),
|
||||
)
|
||||
return events
|
||||
|
||||
@pytest.mark.parametrize("mode", ["visible", "metadata"])
|
||||
def test_one_terminal_event_per_image(self, tmp_path, mode):
|
||||
events = self._run(tmp_path, mode)
|
||||
terminal = [name for name, stage, _ in events if stage in ("done", "failed")]
|
||||
assert sorted(terminal) == ["img0.png", "img1.png", "img2.png"]
|
||||
|
||||
def test_progress_is_a_token_not_prose(self, tmp_path):
|
||||
"""The CLI keys console text off these tokens; free text would break it."""
|
||||
events = self._run(tmp_path, "visible")
|
||||
assert {stage for _, stage, _ in events} <= {"visible", "invisible", "metadata", "done", "failed"}
|
||||
|
||||
@@ -119,6 +119,21 @@ class TestDetectAndMask:
|
||||
img = np.full((1024, 1024, 3), 100, np.uint8)
|
||||
assert eng.footprint_mask(img) is None
|
||||
|
||||
def test_force_masks_the_whole_locate_box_on_a_clean_frame(self):
|
||||
"""``force`` takes priority over detection for this mark, unlike the base
|
||||
policy: a --no-detect caller named the mark, so the honest footprint is the
|
||||
whole geometry box even though nothing was detected."""
|
||||
eng = BaiduEngine()
|
||||
img = np.full((1024, 1024, 3), 100, np.uint8)
|
||||
mask = eng.footprint_mask(img, force=True)
|
||||
assert mask is not None
|
||||
bx, by, bw, bh = eng.locate(img).bbox
|
||||
ys, xs = np.where(mask > 0)
|
||||
assert xs.min() <= bx
|
||||
assert xs.max() >= bx + bw - 1
|
||||
assert ys.min() <= by
|
||||
assert ys.max() >= by + bh - 1
|
||||
|
||||
|
||||
class TestPillInteraction:
|
||||
def test_confident_baidu_detection_suppresses_the_jimeng_pill(self):
|
||||
|
||||
+6
-1
@@ -552,7 +552,9 @@ class TestAllCommand:
|
||||
output = tmp_path / "clean.png"
|
||||
with (
|
||||
patch("remove_ai_watermarks.invisible_engine.is_available", return_value=True),
|
||||
patch("remove_ai_watermarks.cli._should_skip_invisible_scrub", return_value=True),
|
||||
# The no-signal gate lives in the library now; patch it there, not on the
|
||||
# CLI module, or the diffusion stage runs for real.
|
||||
patch("remove_ai_watermarks.api._SourceEvidence.has_invisible_target", return_value=False),
|
||||
patch(
|
||||
"remove_ai_watermarks.metadata.strip_and_verify",
|
||||
return_value=(tmp_path / "intermediate.png", {"c2pa": True}),
|
||||
@@ -564,6 +566,9 @@ class TestAllCommand:
|
||||
assert "metadata" in result.output.lower()
|
||||
assert "survived" in result.output.lower()
|
||||
assert "AI metadata stripped" not in result.output
|
||||
# An incomplete strip must leave NOTHING on disk: an AI-readable output plus a
|
||||
# non-zero exit is the failure mode the pre-write raise exists to prevent.
|
||||
assert not output.exists()
|
||||
|
||||
def test_all_preserves_rgba_across_invisible_step(self, runner, tmp_path):
|
||||
"""Regression: ``all`` must keep transparency even when the invisible
|
||||
|
||||
@@ -18,8 +18,8 @@ from remove_ai_watermarks.doubao_engine import (
|
||||
_alpha_template,
|
||||
_glyph_silhouette,
|
||||
_template_match_score,
|
||||
load_image_bgr,
|
||||
)
|
||||
from remove_ai_watermarks.image_io import load_image_bgr
|
||||
|
||||
SAMPLE = Path(__file__).resolve().parents[1] / "data" / "fixtures" / "provenance" / "doubao-1.png"
|
||||
|
||||
|
||||
@@ -1274,3 +1274,76 @@ class TestRealSamplesHaveNoClash:
|
||||
pytest.skip(f"{fixture} not present")
|
||||
r = identify(path, check_visible=False, check_invisible=False)
|
||||
assert r.integrity_clashes == []
|
||||
|
||||
|
||||
class TestSharedPixelDecode:
|
||||
"""Every pixel detector in one report reads ONE decode of the source.
|
||||
|
||||
``identify`` used to decode the file three times -- the DWT-DCT detector, the
|
||||
TrustMark detector and the visible-mark stage each opened it. TrustMark still
|
||||
keeps its own PIL decode on purpose (cv2 and PIL disagree on EXIF orientation and
|
||||
on 16-bit PNG, so substituting one for the other is not behavior-preserving).
|
||||
"""
|
||||
|
||||
SAMPLE = SAMPLES_DIR / "chatgpt-1.png"
|
||||
|
||||
def _count_source_decodes(self, **kwargs) -> int:
|
||||
from unittest.mock import patch
|
||||
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
real = image_io.imread
|
||||
# Warm the bundled-asset caches first: alpha templates and sparkle captures
|
||||
# also go through imread, and they are one-time loads, not source decodes.
|
||||
identify(self.SAMPLE, check_visible=True, check_invisible=True)
|
||||
seen: list[str] = []
|
||||
|
||||
def counted(path, *args, **kw):
|
||||
seen.append(str(path))
|
||||
return real(path, *args, **kw)
|
||||
|
||||
with patch.object(image_io, "imread", counted):
|
||||
identify(self.SAMPLE, **kwargs)
|
||||
return sum(1 for s in seen if s == str(self.SAMPLE))
|
||||
|
||||
@pytest.mark.skipif(not (SAMPLES_DIR / "chatgpt-1.png").exists(), reason="sample not present")
|
||||
def test_visible_and_invisible_share_one_decode(self):
|
||||
assert self._count_source_decodes(check_visible=True, check_invisible=True) == 1
|
||||
|
||||
@pytest.mark.skipif(not (SAMPLES_DIR / "chatgpt-1.png").exists(), reason="sample not present")
|
||||
def test_metadata_only_report_never_decodes(self):
|
||||
assert self._count_source_decodes(check_visible=False, check_invisible=False) == 0
|
||||
|
||||
@pytest.mark.skipif(not (SAMPLES_DIR / "chatgpt-1.png").exists(), reason="sample not present")
|
||||
def test_a_decode_failure_still_propagates_on_the_invisible_arm(self):
|
||||
"""Load-bearing: ``has_invisible_target`` turns this exception into its
|
||||
documented fail-safe ``True``. Swallowing it would skip a diffusion scrub on a
|
||||
file that used to get one -- a watermark left on a paid removal."""
|
||||
from unittest.mock import patch
|
||||
|
||||
from remove_ai_watermarks import identify as identify_mod
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
def boom(*args, **kwargs):
|
||||
raise OSError("decode exploded")
|
||||
|
||||
with patch.object(image_io, "imread", boom):
|
||||
with pytest.raises(OSError, match="decode exploded"):
|
||||
identify(self.SAMPLE, check_visible=False, check_invisible=True)
|
||||
assert identify_mod.has_invisible_target(self.SAMPLE) is True
|
||||
|
||||
@pytest.mark.skipif(not (SAMPLES_DIR / "chatgpt-1.png").exists(), reason="sample not present")
|
||||
def test_a_decode_failure_is_swallowed_on_the_visible_arm(self):
|
||||
"""The visible arm's historical behavior: no cv2, no visible marks, and the
|
||||
metadata verdict is untouched."""
|
||||
from unittest.mock import patch
|
||||
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
def boom(*args, **kwargs):
|
||||
raise OSError("decode exploded")
|
||||
|
||||
with patch.object(image_io, "imread", boom):
|
||||
report = identify(self.SAMPLE, check_visible=True, check_invisible=False)
|
||||
assert not any(s.name.startswith("visible_") for s in report.signals)
|
||||
assert report.is_ai_generated is True # the C2PA verdict survives the decode failure
|
||||
|
||||
@@ -110,3 +110,26 @@ class TestDetectAndMask:
|
||||
eng = LibLibEngine()
|
||||
img = np.full((2400, 1792, 3), 100, np.uint8)
|
||||
assert eng.footprint_mask(img) is None
|
||||
|
||||
def test_confident_liblib_detection_suppresses_the_jimeng_pill(self):
|
||||
# A LibLibAI image is TC260 too but is not Jimeng-basic: like Doubao/Qwen/
|
||||
# Kling/RunningHub/Baidu, a confident LibLibAI detection must veto the pill.
|
||||
# It was the one mark the hand-written veto list in ``_keep_pill`` missed.
|
||||
from remove_ai_watermarks.watermark_registry import _keep_pill
|
||||
|
||||
assert not _keep_pill({"liblib"}, provenance=frozenset({"jimeng"}), footprint_flat=1.0)
|
||||
|
||||
def test_force_masks_the_whole_locate_box_on_a_clean_frame(self):
|
||||
"""``force`` takes priority over detection for this mark, unlike the base
|
||||
policy: a --no-detect caller named the mark, so the honest footprint is the
|
||||
whole geometry box even though nothing was detected."""
|
||||
eng = LibLibEngine()
|
||||
img = np.full((2400, 1792, 3), 100, np.uint8)
|
||||
mask = eng.footprint_mask(img, force=True)
|
||||
assert mask is not None
|
||||
bx, by, bw, bh = eng.locate(img).bbox
|
||||
ys, xs = np.where(mask > 0)
|
||||
assert xs.min() <= bx
|
||||
assert xs.max() >= bx + bw - 1
|
||||
assert ys.min() <= by
|
||||
assert ys.max() >= by + bh - 1
|
||||
|
||||
@@ -590,6 +590,188 @@ class TestC2paBufferScans:
|
||||
assert synthid_verdict("Google LLC") == "likely present (Google LLC embeds SynthID with C2PA)"
|
||||
|
||||
|
||||
def _amf0_str(value: bytes, *, long: bool = False) -> bytes:
|
||||
marker = b"\x0c" if long else b"\x02"
|
||||
return marker + len(value).to_bytes(4 if long else 2, "big") + value
|
||||
|
||||
|
||||
def _amf0_property(name: bytes, value: bytes) -> bytes:
|
||||
return len(name).to_bytes(2, "big") + name + value
|
||||
|
||||
|
||||
_AMF0_OBJECT_END = b"\x00\x00\x09"
|
||||
|
||||
# A minimal TC260-PG-20257A label: the reader validates the JSON before accepting it,
|
||||
# so the walker tests need a value that actually parses.
|
||||
_TC260_AIGC_VALUE = (
|
||||
b'{"Label":"1","ContentProducer":"00119144030008867405X210002",'
|
||||
b'"ProduceID":"sample-001","ReservedCode1":"","ContentPropagator":"",'
|
||||
b'"PropagateID":"","ReservedCode2":""}'
|
||||
)
|
||||
|
||||
|
||||
class TestFlvAmf0Walker:
|
||||
"""``_skip_amf0`` is what lets the FLV reader step over every property that is not
|
||||
``AIGC``. Each AMF0 type it does not walk correctly aborts the scan, so a label that
|
||||
sits after an unhandled type is silently missed. Pure byte parsing -- no media file
|
||||
and no decoder is involved, so every branch is reachable from synthetic bytes."""
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("name", "encoded"),
|
||||
[
|
||||
("number", b"\x00" + b"\x00" * 8),
|
||||
("boolean", b"\x01\x01"),
|
||||
("string", _amf0_str(b"a string")),
|
||||
("null", b"\x05"),
|
||||
("undefined", b"\x06"),
|
||||
("reference", b"\x07\x00\x01"),
|
||||
("date", b"\x0b" + b"\x00" * 10),
|
||||
("long-string", _amf0_str(b"a long string", long=True)),
|
||||
("strict-array", b"\x0a\x00\x00\x00\x02" + b"\x00" + b"\x00" * 8 + b"\x01\x00"),
|
||||
("object", b"\x03" + _amf0_property(b"inner", b"\x01\x00") + _AMF0_OBJECT_END),
|
||||
("ecma-array", b"\x08\x00\x00\x00\x01" + _amf0_property(b"inner", b"\x05") + _AMF0_OBJECT_END),
|
||||
],
|
||||
)
|
||||
def test_every_walkable_type_is_stepped_over(self, name: str, encoded: bytes):
|
||||
"""A property of this type, sitting before the AIGC one, must not stop the walk."""
|
||||
from remove_ai_watermarks._internal.flv import _script_payloads
|
||||
|
||||
payload = (
|
||||
_amf0_str(b"onMetaData")
|
||||
+ b"\x03"
|
||||
+ _amf0_property(name.encode(), encoded)
|
||||
+ _amf0_property(b"AIGC", _amf0_str(_TC260_AIGC_VALUE))
|
||||
+ _AMF0_OBJECT_END
|
||||
)
|
||||
assert _script_payloads(payload) == (_TC260_AIGC_VALUE,)
|
||||
|
||||
def test_unknown_type_marker_stops_the_walk(self):
|
||||
"""An unrecognized marker has an unknown width, so the reader cannot guess where
|
||||
the next property starts. It must give up rather than resynchronize on garbage."""
|
||||
from remove_ai_watermarks._internal.flv import _script_payloads
|
||||
|
||||
payload = (
|
||||
_amf0_str(b"onMetaData")
|
||||
+ b"\x03"
|
||||
+ _amf0_property(b"mystery", b"\x7f")
|
||||
+ _amf0_property(b"AIGC", _amf0_str(_TC260_AIGC_VALUE))
|
||||
+ _AMF0_OBJECT_END
|
||||
)
|
||||
assert _script_payloads(payload) == ()
|
||||
|
||||
def test_truncated_value_stops_the_walk(self):
|
||||
"""A declared length running past the buffer must return empty, not raise."""
|
||||
from remove_ai_watermarks._internal.flv import _script_payloads
|
||||
|
||||
payload = _amf0_str(b"onMetaData") + b"\x03" + _amf0_property(b"trunc", b"\x02\x00\xff")
|
||||
assert _script_payloads(payload) == ()
|
||||
|
||||
def test_nesting_deeper_than_the_depth_cap_is_refused(self):
|
||||
"""The depth cap bounds work on hostile input; past it the walker returns None."""
|
||||
from remove_ai_watermarks._internal.flv import _skip_amf0
|
||||
|
||||
nested = b"\x05"
|
||||
for _ in range(12):
|
||||
nested = b"\x03" + _amf0_property(b"n", nested) + _AMF0_OBJECT_END
|
||||
assert _skip_amf0(nested, 0) is None
|
||||
|
||||
def test_long_string_aigc_value_is_read(self):
|
||||
"""TC260 values large enough to need the 4-byte long-string form still parse."""
|
||||
from remove_ai_watermarks._internal.flv import _script_payloads
|
||||
|
||||
payload = (
|
||||
_amf0_str(b"onMetaData")
|
||||
+ b"\x03"
|
||||
+ _amf0_property(b"AIGC", _amf0_str(_TC260_AIGC_VALUE, long=True))
|
||||
+ _AMF0_OBJECT_END
|
||||
)
|
||||
assert _script_payloads(payload) == (_TC260_AIGC_VALUE,)
|
||||
|
||||
def test_non_onmetadata_script_tag_is_ignored(self):
|
||||
"""Only ``onMetaData`` carries the normative label; other script tags are skipped."""
|
||||
from remove_ai_watermarks._internal.flv import _script_payloads
|
||||
|
||||
payload = (
|
||||
_amf0_str(b"onCuePoint")
|
||||
+ b"\x03"
|
||||
+ _amf0_property(b"AIGC", _amf0_str(_TC260_AIGC_VALUE))
|
||||
+ _AMF0_OBJECT_END
|
||||
)
|
||||
assert _script_payloads(payload) == ()
|
||||
|
||||
def test_missing_file_reads_as_no_payloads(self, tmp_path: Path):
|
||||
from remove_ai_watermarks._internal.flv import tc260_aigc_payloads
|
||||
|
||||
assert tc260_aigc_payloads(tmp_path / "absent.flv") == ()
|
||||
|
||||
def test_non_flv_signature_reads_as_no_payloads(self, tmp_path: Path):
|
||||
from remove_ai_watermarks._internal.flv import tc260_aigc_payloads
|
||||
|
||||
path = tmp_path / "fake.flv"
|
||||
path.write_bytes(b"NOTFLV\x00\x00\x09" + b"\x00" * 32)
|
||||
assert tc260_aigc_payloads(path) == ()
|
||||
|
||||
|
||||
class TestProbeMemoization:
|
||||
"""The per-file probes are cached on (path, mtime_ns, size).
|
||||
|
||||
Their only real failure mode is staleness after an IN-PLACE rewrite, which this
|
||||
package does (``remove_ai_metadata(p, p)``, and the batch case where the output
|
||||
directory is the input directory). mtime alone can land inside one tick on a
|
||||
coarse filesystem, hence size in the key too.
|
||||
"""
|
||||
|
||||
def test_in_place_strip_invalidates_the_label_cache(self, tmp_path: Path):
|
||||
import shutil
|
||||
|
||||
from remove_ai_watermarks.metadata import aigc_label, remove_ai_metadata
|
||||
|
||||
source = Path(__file__).resolve().parents[1] / "data" / "fixtures" / "provenance" / "doubao-1.png"
|
||||
if not source.exists():
|
||||
pytest.skip("doubao sample not present")
|
||||
target = tmp_path / "in_place.png"
|
||||
shutil.copyfile(source, target)
|
||||
|
||||
assert aigc_label(target) is not None # populates the cache
|
||||
remove_ai_metadata(target, target)
|
||||
assert aigc_label(target) is None, "the cache answered from the pre-strip content"
|
||||
|
||||
def test_caller_cannot_mutate_the_cached_label(self, tmp_path: Path):
|
||||
"""``aigc_label`` returns a dict; a caller editing it must not poison the cache."""
|
||||
import shutil
|
||||
|
||||
from remove_ai_watermarks.metadata import aigc_label
|
||||
|
||||
source = Path(__file__).resolve().parents[1] / "data" / "fixtures" / "provenance" / "doubao-1.png"
|
||||
if not source.exists():
|
||||
pytest.skip("doubao sample not present")
|
||||
target = tmp_path / "mutate.png"
|
||||
shutil.copyfile(source, target)
|
||||
|
||||
first = aigc_label(target)
|
||||
assert first is not None
|
||||
first["ContentProducer"] = "TAMPERED"
|
||||
second = aigc_label(target)
|
||||
assert second is not None
|
||||
assert second["ContentProducer"] != "TAMPERED"
|
||||
|
||||
def test_unstattable_path_bypasses_the_cache_and_behaves_as_before(self, tmp_path: Path):
|
||||
"""A path that cannot be stat'ed has no cache key, so it must fall through to the
|
||||
uncached implementation -- same outcome as before memoization, whatever that is."""
|
||||
from remove_ai_watermarks import metadata
|
||||
|
||||
missing = tmp_path / "absent.png"
|
||||
assert metadata._stat_key(missing) is None
|
||||
|
||||
def outcome(fn):
|
||||
try:
|
||||
return ("value", fn(missing))
|
||||
except Exception as exc:
|
||||
return ("raised", type(exc).__name__)
|
||||
|
||||
assert outcome(metadata.aigc_label) == outcome(metadata._aigc_label_impl)
|
||||
|
||||
|
||||
class TestC2PAInvalidSignature:
|
||||
"""A .png file that is not actually PNG-signed must read as clean, not crash."""
|
||||
|
||||
@@ -602,3 +784,54 @@ class TestC2PAInvalidSignature:
|
||||
fake = tmp_path / "fake.png"
|
||||
fake.write_bytes(b"\xff\xd8\xff\xe0 not a png at all, just garbage bytes")
|
||||
assert extract_c2pa_chunk(fake) is None
|
||||
|
||||
|
||||
class TestTc260ContainerRouting:
|
||||
"""The native-container readers route on CONTENT, not on the file extension.
|
||||
|
||||
Every reader self-gates on its own magic bytes after a 4-12 byte read, so gating
|
||||
the AVI and FLV ones on the suffix as well was redundant -- and it made a
|
||||
correctly-formatted container served under the wrong name invisible, contradicting
|
||||
this module's own rule that format detection reads the bytes.
|
||||
"""
|
||||
|
||||
@staticmethod
|
||||
def _riff_chunk(chunk_id: bytes, payload: bytes) -> bytes:
|
||||
return chunk_id + len(payload).to_bytes(4, "little") + payload + (b"\x00" if len(payload) & 1 else b"")
|
||||
|
||||
def _labelled_avi(self) -> bytes:
|
||||
info = self._riff_chunk(b"AIGC", _TC260_AIGC_VALUE)
|
||||
body = b"AVI " + self._riff_chunk(b"LIST", b"INFO" + info)
|
||||
return b"RIFF" + len(body).to_bytes(4, "little") + body
|
||||
|
||||
def test_a_mislabeled_avi_is_still_read(self, tmp_path: Path):
|
||||
from remove_ai_watermarks.metadata import aigc_label
|
||||
|
||||
target = tmp_path / "clip.bin" # correct AVI bytes, wrong suffix
|
||||
target.write_bytes(self._labelled_avi())
|
||||
label = aigc_label(target)
|
||||
assert label is not None
|
||||
assert label["Label"] == "1"
|
||||
|
||||
def test_a_correctly_named_avi_still_works(self, tmp_path: Path):
|
||||
from remove_ai_watermarks.metadata import aigc_label
|
||||
|
||||
target = tmp_path / "clip.avi"
|
||||
target.write_bytes(self._labelled_avi())
|
||||
assert aigc_label(target) is not None
|
||||
|
||||
def test_webp_yields_nothing_from_the_riff_reader(self, tmp_path: Path):
|
||||
"""WebP is the one input class the now-unconditional RIFF reader newly touches,
|
||||
and it shares the ``RIFF`` prefix -- the ``AVI `` form check is what rejects it."""
|
||||
from remove_ai_watermarks._internal.riff import tc260_aigc_payloads
|
||||
|
||||
body = b"WEBP" + self._riff_chunk(b"VP8L", b"\x00" * 16)
|
||||
target = tmp_path / "pic.webp"
|
||||
target.write_bytes(b"RIFF" + len(body).to_bytes(4, "little") + body)
|
||||
assert tc260_aigc_payloads(target) == ()
|
||||
|
||||
def test_every_reader_is_reached_in_a_stable_order(self):
|
||||
from remove_ai_watermarks.metadata import _tc260_container_readers
|
||||
|
||||
readers = _tc260_container_readers()
|
||||
assert [r.__module__.rsplit(".", 1)[-1] for r in readers] == ["isobmff", "ebml", "riff", "flv"]
|
||||
|
||||
@@ -113,7 +113,15 @@ class TestPillGate:
|
||||
flatness matters, so tests pass a flat or textured frame."""
|
||||
|
||||
@staticmethod
|
||||
def _fakes(monkeypatch: pytest.MonkeyPatch, keys: set[str]) -> None:
|
||||
def _fakes(monkeypatch: pytest.MonkeyPatch, keys: set[str]) -> list[str]:
|
||||
"""Fake every mark's verdict. Returns the list the fake appends each call to.
|
||||
|
||||
BOTH entry points must be faked: the arbiter's perception pass goes through
|
||||
``detect_both`` (one scan, two verdicts) while the removal path still calls
|
||||
``detect``. Patching only one leaves the real detectors running on a synthetic
|
||||
frame, where they find nothing -- so a "dropped" assertion would pass for
|
||||
entirely the wrong reason. The returned call log is what pins that.
|
||||
"""
|
||||
from remove_ai_watermarks.watermark_registry import KnownMark, MarkDetection
|
||||
|
||||
labels = {
|
||||
@@ -122,13 +130,21 @@ class TestPillGate:
|
||||
"jimeng_pill": "Jimeng AI生成 pill",
|
||||
}
|
||||
monkeypatch.setattr(registry, "preferred_inpaint_backend", lambda: "cv2")
|
||||
calls: list[str] = []
|
||||
|
||||
def fake_detect(self: KnownMark, image: object, *, provenance: bool = False) -> MarkDetection:
|
||||
calls.append(self.key)
|
||||
return MarkDetection(
|
||||
self.key, labels.get(self.key, self.key), "loc", self.key in keys, 0.6, (10, 10, 40, 40)
|
||||
)
|
||||
|
||||
def fake_detect_both(self: KnownMark, image: object) -> tuple[MarkDetection, MarkDetection]:
|
||||
d = fake_detect(self, image)
|
||||
return d, d
|
||||
|
||||
monkeypatch.setattr(registry.KnownMark, "detect", fake_detect)
|
||||
monkeypatch.setattr(registry.KnownMark, "detect_both", fake_detect_both)
|
||||
return calls
|
||||
|
||||
def test_pill_kept_with_metadata_on_flat_footprint(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
# jimeng provenance (TC260) + flat background -> safe fill, remove
|
||||
@@ -138,8 +154,9 @@ class TestPillGate:
|
||||
|
||||
def test_pill_dropped_with_metadata_on_textured_footprint(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
# jimeng provenance + textured background (ceiling-like) -> fill would smear, skip
|
||||
self._fakes(monkeypatch, {"jimeng_pill"})
|
||||
calls = self._fakes(monkeypatch, {"jimeng_pill"})
|
||||
_, removed = registry.remove_auto_marks(_textured_frame(), provenance=frozenset({"jimeng"}))
|
||||
assert "jimeng_pill" in calls, "the fake detector never ran -- this would pass vacuously"
|
||||
assert "Jimeng AI生成 pill" not in removed
|
||||
|
||||
def test_pill_kept_via_wordmark_ignores_texture(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
@@ -151,8 +168,9 @@ class TestPillGate:
|
||||
|
||||
def test_pill_dropped_on_textured_footprint(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
# The metadata arm keeps the flatness guard: textured false fires visibly smear.
|
||||
self._fakes(monkeypatch, {"jimeng_pill"})
|
||||
calls = self._fakes(monkeypatch, {"jimeng_pill"})
|
||||
_, removed = registry.remove_auto_marks(_textured_frame(), provenance=frozenset({"jimeng"}))
|
||||
assert "jimeng_pill" in calls, "the fake detector never ran -- this would pass vacuously"
|
||||
assert "Jimeng AI生成 pill" not in removed
|
||||
|
||||
def test_pill_dropped_without_metadata_or_wordmark(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
|
||||
@@ -117,6 +117,46 @@ class TestNonBgrInputs:
|
||||
assert np.array_equal(out[..., 3], bgra[..., 3])
|
||||
|
||||
|
||||
class TestBackendTable:
|
||||
"""The fill-backend names are stated in several places; they must agree.
|
||||
|
||||
They are deliberately separate LITERALS rather than one derived list: deriving the
|
||||
CLI choices or the registry's ``Backend`` from ``region_eraser`` would give
|
||||
``watermark_registry`` (and therefore every ``--help`` and every metadata-only
|
||||
``identify``) a module-level cv2 import. This test keeps the copies in sync instead.
|
||||
"""
|
||||
|
||||
def test_registry_literal_matches_the_eraser_table(self):
|
||||
import typing
|
||||
|
||||
from remove_ai_watermarks import region_eraser, watermark_registry
|
||||
|
||||
assert set(typing.get_args(watermark_registry.Backend)) == set(region_eraser.FILL_BACKENDS)
|
||||
|
||||
def test_executable_backends_are_the_table_minus_auto(self):
|
||||
import typing
|
||||
|
||||
from remove_ai_watermarks import region_eraser
|
||||
|
||||
assert set(typing.get_args(region_eraser.Backend)) == set(region_eraser.FILL_BACKENDS) - {"auto"}
|
||||
|
||||
def test_learned_backends_name_real_module_attributes(self):
|
||||
from remove_ai_watermarks import region_eraser
|
||||
|
||||
for name, row in region_eraser._LEARNED_BACKENDS.items():
|
||||
assert name in region_eraser.FILL_BACKENDS
|
||||
assert callable(getattr(region_eraser, row.available))
|
||||
assert callable(getattr(region_eraser, row.erase))
|
||||
|
||||
def test_unknown_backend_degrades_to_cv2_instead_of_raising(self):
|
||||
"""``erase`` is public: a library caller passing ``"auto"`` (or a typo) must get
|
||||
the classical fill, not a KeyError."""
|
||||
img = np.full((64, 64, 3), 100, np.uint8)
|
||||
mask = np.zeros((64, 64), np.uint8)
|
||||
mask[20:40, 20:40] = 255
|
||||
assert erase(img, mask=mask, backend="auto").shape == img.shape # type: ignore[arg-type]
|
||||
|
||||
|
||||
class TestLamaBackend:
|
||||
def test_lama_raises_when_unavailable(self):
|
||||
img = np.full((100, 100, 3), 50, np.uint8)
|
||||
|
||||
+51
-48
@@ -1625,14 +1625,14 @@ class TestSoraTemporalArbiter:
|
||||
_BOX = (40, 60, 150, 54)
|
||||
|
||||
def test_four_frame_lookalike_run_is_too_short(self):
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_sora_localizations
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_localizations
|
||||
|
||||
detections = [FrameLocalization(index, 0.70, self._BOX) for index in range(4)]
|
||||
|
||||
assert stabilize_sora_localizations(detections, provenance=False) == [None] * 4
|
||||
assert stabilize_localizations("sora", detections, provenance=False) == [None] * 4
|
||||
|
||||
def test_provenance_accepts_recurring_low_contrast_visual_match(self):
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_sora_localizations
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_localizations
|
||||
|
||||
detections = [
|
||||
FrameLocalization(0, 0.59, self._BOX),
|
||||
@@ -1642,10 +1642,10 @@ class TestSoraTemporalArbiter:
|
||||
FrameLocalization(4, 0.61, self._BOX),
|
||||
]
|
||||
|
||||
assert stabilize_sora_localizations(detections, provenance=True) == [self._BOX] * 5
|
||||
assert stabilize_localizations("sora", detections, provenance=True) == [self._BOX] * 5
|
||||
|
||||
def test_confirmed_provenance_run_covers_transition_frames(self):
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_sora_localizations
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_localizations
|
||||
|
||||
detections = [
|
||||
FrameLocalization(0, 0.30, (500, 300, 54, 54)),
|
||||
@@ -1657,10 +1657,10 @@ class TestSoraTemporalArbiter:
|
||||
FrameLocalization(6, 0.30, (300, 100, 54, 54)),
|
||||
]
|
||||
|
||||
assert stabilize_sora_localizations(detections, provenance=True) == [self._BOX] * 7
|
||||
assert stabilize_localizations("sora", detections, provenance=True) == [self._BOX] * 7
|
||||
|
||||
def test_transition_prefers_low_score_match_at_a_confirmed_position(self):
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_sora_localizations
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_localizations
|
||||
|
||||
other_box = (500, 300, 150, 54)
|
||||
detections = [
|
||||
@@ -1678,13 +1678,13 @@ class TestSoraTemporalArbiter:
|
||||
FrameLocalization(11, 0.62, other_box),
|
||||
]
|
||||
|
||||
stabilized = stabilize_sora_localizations(detections, provenance=True)
|
||||
stabilized = stabilize_localizations("sora", detections, provenance=True)
|
||||
|
||||
assert stabilized[6] == self._BOX
|
||||
assert stabilized[7:] == [other_box] * 5
|
||||
|
||||
def test_transition_without_a_match_keeps_previous_stable_position(self):
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_sora_localizations
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_localizations
|
||||
|
||||
other_box = (500, 300, 150, 54)
|
||||
detections = [
|
||||
@@ -1702,12 +1702,12 @@ class TestSoraTemporalArbiter:
|
||||
FrameLocalization(11, 0.62, other_box),
|
||||
]
|
||||
|
||||
stabilized = stabilize_sora_localizations(detections, provenance=True)
|
||||
stabilized = stabilize_localizations("sora", detections, provenance=True)
|
||||
|
||||
assert stabilized[5:7] == [self._BOX, self._BOX]
|
||||
|
||||
def test_unproven_weak_run_is_rejected(self):
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_sora_localizations
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_localizations
|
||||
|
||||
detections = [
|
||||
FrameLocalization(0, 0.61, self._BOX),
|
||||
@@ -1717,10 +1717,10 @@ class TestSoraTemporalArbiter:
|
||||
FrameLocalization(4, 0.61, self._BOX),
|
||||
]
|
||||
|
||||
assert stabilize_sora_localizations(detections, provenance=False) == [None] * 5
|
||||
assert stabilize_localizations("sora", detections, provenance=False) == [None] * 5
|
||||
|
||||
def test_strong_recurring_visual_run_needs_no_metadata(self):
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_sora_localizations
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_localizations
|
||||
|
||||
detections = [
|
||||
FrameLocalization(0, 0.61, self._BOX),
|
||||
@@ -1730,10 +1730,10 @@ class TestSoraTemporalArbiter:
|
||||
FrameLocalization(4, 0.62, self._BOX),
|
||||
]
|
||||
|
||||
assert stabilize_sora_localizations(detections, provenance=False) == [self._BOX] * 5
|
||||
assert stabilize_localizations("sora", detections, provenance=False) == [self._BOX] * 5
|
||||
|
||||
def test_isolated_lookalikes_at_different_positions_are_rejected(self):
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_sora_localizations
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_localizations
|
||||
|
||||
detections = [
|
||||
FrameLocalization(0, 0.70, (10, 10, 150, 54)),
|
||||
@@ -1741,10 +1741,10 @@ class TestSoraTemporalArbiter:
|
||||
FrameLocalization(2, 0.70, (650, 400, 150, 54)),
|
||||
]
|
||||
|
||||
assert stabilize_sora_localizations(detections, provenance=True) == [None, None, None]
|
||||
assert stabilize_localizations("sora", detections, provenance=True) == [None, None, None]
|
||||
|
||||
def test_short_dropout_between_matching_boxes_is_filled(self):
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_sora_localizations
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_localizations
|
||||
|
||||
detections = [
|
||||
FrameLocalization(0, 0.66, self._BOX),
|
||||
@@ -1755,40 +1755,40 @@ class TestSoraTemporalArbiter:
|
||||
FrameLocalization(5, 0.66, self._BOX),
|
||||
]
|
||||
|
||||
assert stabilize_sora_localizations(detections, provenance=False) == [self._BOX] * 6
|
||||
assert stabilize_localizations("sora", detections, provenance=False) == [self._BOX] * 6
|
||||
|
||||
|
||||
class TestVeoTemporalArbiter:
|
||||
_BOX = (1132, 572, 56, 56)
|
||||
|
||||
def test_eleven_frame_lookalike_run_is_too_short(self):
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_veo_localizations
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_localizations
|
||||
|
||||
detections = [FrameLocalization(index, 0.70, self._BOX) for index in range(11)]
|
||||
|
||||
assert stabilize_veo_localizations(detections, provenance=False) == [None] * 11
|
||||
assert stabilize_localizations("veo", detections, provenance=False) == [None] * 11
|
||||
|
||||
def test_strong_fixed_run_covers_video_without_metadata(self):
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_veo_localizations
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_localizations
|
||||
|
||||
detections = [FrameLocalization(index, 0.60, self._BOX) for index in range(12)]
|
||||
detections.extend(FrameLocalization(index, 0.20, (300, 200, 48, 48)) for index in range(12, 15))
|
||||
|
||||
assert stabilize_veo_localizations(detections, provenance=False) == [self._BOX] * 15
|
||||
assert stabilize_localizations("veo", detections, provenance=False) == [self._BOX] * 15
|
||||
|
||||
def test_google_provenance_accepts_recurring_low_contrast_diamond(self):
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_veo_localizations
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_localizations
|
||||
|
||||
detections = [FrameLocalization(index, 0.47, self._BOX) for index in range(12)]
|
||||
|
||||
assert stabilize_veo_localizations(detections, provenance=True) == [self._BOX] * 12
|
||||
assert stabilize_localizations("veo", detections, provenance=True) == [self._BOX] * 12
|
||||
|
||||
def test_unproven_weak_run_is_rejected(self):
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_veo_localizations
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_localizations
|
||||
|
||||
detections = [FrameLocalization(index, 0.52, self._BOX) for index in range(12)]
|
||||
|
||||
assert stabilize_veo_localizations(detections, provenance=False) == [None] * 12
|
||||
assert stabilize_localizations("veo", detections, provenance=False) == [None] * 12
|
||||
|
||||
|
||||
class TestByteDanceTemporalArbiter:
|
||||
@@ -1796,47 +1796,47 @@ class TestByteDanceTemporalArbiter:
|
||||
_DOLA_BOX = (1160, 680, 96, 22)
|
||||
|
||||
def test_seedance_requires_twelve_recurring_frames(self):
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_seedance_localizations
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_localizations
|
||||
|
||||
detections = [FrameLocalization(index, 0.50, self._SEEDANCE_BOX) for index in range(11)]
|
||||
|
||||
assert stabilize_seedance_localizations(detections, provenance=False) == [None] * 11
|
||||
assert stabilize_localizations("seedance", detections, provenance=False) == [None] * 11
|
||||
|
||||
def test_seedance_strong_run_covers_low_contrast_frames(self):
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_seedance_localizations
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_localizations
|
||||
|
||||
detections = [FrameLocalization(index, 0.45, self._SEEDANCE_BOX) for index in range(12)]
|
||||
detections.extend(FrameLocalization(index, 0.20, (200, 100, 80, 60)) for index in range(12, 15))
|
||||
|
||||
assert stabilize_seedance_localizations(detections, provenance=False) == [self._SEEDANCE_BOX] * 15
|
||||
assert stabilize_localizations("seedance", detections, provenance=False) == [self._SEEDANCE_BOX] * 15
|
||||
|
||||
def test_seedance_rejects_a_slowly_drifting_scene_detail(self):
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_seedance_localizations
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_localizations
|
||||
|
||||
detections = [FrameLocalization(index, 0.46, (1110 - index * 3, 610, 90, 66)) for index in range(14)]
|
||||
|
||||
assert stabilize_seedance_localizations(detections, provenance=False) == [None] * 14
|
||||
assert stabilize_localizations("seedance", detections, provenance=False) == [None] * 14
|
||||
|
||||
def test_dola_requires_twelve_recurring_frames(self):
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_dola_localizations
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_localizations
|
||||
|
||||
detections = [FrameLocalization(index, 0.60, self._DOLA_BOX) for index in range(11)]
|
||||
|
||||
assert stabilize_dola_localizations(detections, provenance=True) == [None] * 11
|
||||
assert stabilize_localizations("dola", detections, provenance=True) == [None] * 11
|
||||
|
||||
def test_dola_provenance_accepts_recurring_low_contrast_text(self):
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_dola_localizations
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_localizations
|
||||
|
||||
detections = [FrameLocalization(index, 0.49, self._DOLA_BOX) for index in range(12)]
|
||||
|
||||
assert stabilize_dola_localizations(detections, provenance=True) == [self._DOLA_BOX] * 12
|
||||
assert stabilize_localizations("dola", detections, provenance=True) == [self._DOLA_BOX] * 12
|
||||
|
||||
def test_dola_without_provenance_needs_a_strong_frame(self):
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_dola_localizations
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_localizations
|
||||
|
||||
detections = [FrameLocalization(index, 0.51, self._DOLA_BOX) for index in range(12)]
|
||||
|
||||
assert stabilize_dola_localizations(detections, provenance=False) == [None] * 12
|
||||
assert stabilize_localizations("dola", detections, provenance=False) == [None] * 12
|
||||
|
||||
def test_bytedance_provenance_requires_ai_source_type(self):
|
||||
from remove_ai_watermarks.video_visible import has_bytedance_video_provenance
|
||||
@@ -1855,23 +1855,24 @@ class TestAdditionalProviderTemporalArbiter:
|
||||
_KLING_BOX = (1018, 664, 245, 42)
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("stabilizer_name", "box", "weak_score", "strong_score"),
|
||||
("mark", "box", "weak_score", "strong_score"),
|
||||
[
|
||||
("stabilize_hailuo_localizations", _HAILUO_BOX, 0.31, 0.35),
|
||||
("stabilize_kling_localizations", _KLING_BOX, 0.21, 0.25),
|
||||
("hailuo", _HAILUO_BOX, 0.31, 0.35),
|
||||
("kling", _KLING_BOX, 0.21, 0.25),
|
||||
],
|
||||
)
|
||||
def test_requires_a_strong_anchored_twelve_frame_run(
|
||||
self,
|
||||
stabilizer_name: str,
|
||||
mark: str,
|
||||
box: tuple[int, int, int, int],
|
||||
weak_score: float,
|
||||
strong_score: float,
|
||||
):
|
||||
from remove_ai_watermarks import video_visible
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization
|
||||
from remove_ai_watermarks.video_visible import FrameLocalization, stabilize_localizations
|
||||
|
||||
def stabilize(dets):
|
||||
return stabilize_localizations(mark, dets)
|
||||
|
||||
stabilize = getattr(video_visible, stabilizer_name)
|
||||
weak = [FrameLocalization(index, weak_score, box) for index in range(12)]
|
||||
strong = [FrameLocalization(index, strong_score, box) for index in range(12)]
|
||||
|
||||
@@ -2165,7 +2166,7 @@ class TestVideoVisibleFullClip:
|
||||
def test_removes_complete_clip_and_preserves_sequence_and_audio(self, tmp_path: Path):
|
||||
from remove_ai_watermarks.metadata import get_ai_metadata
|
||||
from remove_ai_watermarks.video import remove_video_all, remove_video_metadata, remove_video_visible
|
||||
from remove_ai_watermarks.video_visible import scan_video_marks, stabilize_sora_localizations
|
||||
from remove_ai_watermarks.video_visible import scan_video_marks, stabilize_localizations
|
||||
|
||||
ffmpeg, ffprobe = _ffmpeg_test_tools()
|
||||
frame_count = 18
|
||||
@@ -2273,14 +2274,16 @@ class TestVideoVisibleFullClip:
|
||||
all_output_scan = scan_video_marks(all_output, ("sora",))["sora"]
|
||||
assert all(
|
||||
region is None
|
||||
for region in stabilize_sora_localizations(
|
||||
for region in stabilize_localizations(
|
||||
"sora",
|
||||
output_scan.detections,
|
||||
provenance=False,
|
||||
)
|
||||
)
|
||||
assert all(
|
||||
region is None
|
||||
for region in stabilize_sora_localizations(
|
||||
for region in stabilize_localizations(
|
||||
"sora",
|
||||
all_output_scan.detections,
|
||||
provenance=False,
|
||||
)
|
||||
|
||||
@@ -51,6 +51,52 @@ class TestCatalog:
|
||||
with pytest.raises(KeyError):
|
||||
reg.get_mark("nope")
|
||||
|
||||
def test_every_registered_key_has_an_engine_row(self):
|
||||
assert set(reg._ENGINE_CLASS) == set(reg.mark_keys())
|
||||
|
||||
def test_unknown_engine_key_raises(self):
|
||||
with pytest.raises(KeyError):
|
||||
reg._engine("nope")
|
||||
|
||||
|
||||
class TestEngineImportsStayLazy:
|
||||
"""The engine table holds NAMES, not imports.
|
||||
|
||||
``identify`` imports ``watermark_registry`` at module scope for a metadata-only
|
||||
scan, and the engines pull cv2. If the table were ever built from real imports,
|
||||
every ``--help`` and every metadata-only ``identify`` would pay for cv2 -- the
|
||||
exact regression a name table can silently introduce. Run in a subprocess because
|
||||
the in-process ``sys.modules`` is already polluted by the rest of the suite.
|
||||
"""
|
||||
|
||||
def _modules_after(self, statements: str) -> set[str]:
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
script = f"import sys, json\n{statements}\nprint(json.dumps(sorted(sys.modules)))"
|
||||
out = subprocess.run( # noqa: S603 - argv is this interpreter plus a literal script
|
||||
[sys.executable, "-c", script], capture_output=True, text=True, check=True
|
||||
)
|
||||
return set(json.loads(out.stdout))
|
||||
|
||||
def test_importing_the_registry_pulls_no_engine_and_no_cv2(self):
|
||||
loaded = self._modules_after("import remove_ai_watermarks.watermark_registry")
|
||||
assert "remove_ai_watermarks.gemini_engine" not in loaded
|
||||
assert "remove_ai_watermarks.doubao_engine" not in loaded
|
||||
assert "cv2" not in loaded
|
||||
|
||||
def test_importing_identify_pulls_no_engine_and_no_cv2(self):
|
||||
# identify.py imports watermark_registry at module scope, so this -- not the
|
||||
# registry alone -- is the real dependency-light surface.
|
||||
loaded = self._modules_after("import remove_ai_watermarks.identify")
|
||||
assert "remove_ai_watermarks.gemini_engine" not in loaded
|
||||
assert "cv2" not in loaded
|
||||
|
||||
def test_resolving_an_engine_imports_it(self):
|
||||
loaded = self._modules_after("from remove_ai_watermarks import watermark_registry as r\nr._engine('gemini')")
|
||||
assert "remove_ai_watermarks.gemini_engine" in loaded
|
||||
|
||||
|
||||
class TestScan:
|
||||
def test_detect_marks_scans_all(self):
|
||||
@@ -410,6 +456,149 @@ class TestArbiter:
|
||||
assert fired == {"jimeng", "jimeng_pill"}
|
||||
|
||||
|
||||
class TestSinglePassPerception:
|
||||
"""``detect_both`` must equal two ``detect`` calls, for strictly less work.
|
||||
|
||||
The arbiter's perception pass runs every detector at both trust levels. The trust
|
||||
level only moves a threshold (Gemini: whether a false-positive gate demotes the
|
||||
result afterwards), never the measurement -- so the expensive scan is shared. If
|
||||
that ever stops being true, this test is what says so.
|
||||
"""
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"image",
|
||||
[
|
||||
pytest.param(np.full((256, 256, 3), 100, np.uint8), id="clean"),
|
||||
pytest.param(np.zeros((40, 40, 3), np.uint8), id="below-the-size-floor"),
|
||||
],
|
||||
)
|
||||
def test_dual_verdict_matches_two_separate_detects(self, image):
|
||||
for m in reg.known_marks():
|
||||
strict, relaxed = m.detect_both(image)
|
||||
for one, two, level in (
|
||||
(strict, m.detect(image, provenance=False), "strict"),
|
||||
(relaxed, m.detect(image, provenance=True), "relaxed"),
|
||||
):
|
||||
assert one.detected == two.detected, f"{m.key} {level}"
|
||||
assert one.confidence == two.confidence, f"{m.key} {level}"
|
||||
assert one.region == two.region, f"{m.key} {level}"
|
||||
|
||||
@pytest.mark.skipif(not DOUBAO_SAMPLE.exists(), reason="doubao sample not present")
|
||||
def test_dual_verdict_matches_on_a_real_positive(self):
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
image = image_io.imread(DOUBAO_SAMPLE)
|
||||
for m in reg.known_marks():
|
||||
strict, relaxed = m.detect_both(image)
|
||||
assert (strict.detected, strict.confidence) == (
|
||||
m.detect(image, provenance=False).detected,
|
||||
m.detect(image, provenance=False).confidence,
|
||||
), m.key
|
||||
assert (relaxed.detected, relaxed.confidence) == (
|
||||
m.detect(image, provenance=True).detected,
|
||||
m.detect(image, provenance=True).confidence,
|
||||
), m.key
|
||||
|
||||
@pytest.mark.skipif(not DOUBAO_SAMPLE.exists(), reason="doubao sample not present")
|
||||
def test_perception_scans_each_mark_once(self, monkeypatch: pytest.MonkeyPatch):
|
||||
"""The whole point: one template sweep per mark, not two."""
|
||||
import cv2
|
||||
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
image = image_io.imread(DOUBAO_SAMPLE)
|
||||
calls = [0]
|
||||
real = cv2.matchTemplate
|
||||
|
||||
def counted(*args, **kwargs):
|
||||
calls[0] += 1
|
||||
return real(*args, **kwargs)
|
||||
|
||||
monkeypatch.setattr(cv2, "matchTemplate", counted)
|
||||
reg._build_candidates(image)
|
||||
single = calls[0]
|
||||
calls[0] = 0
|
||||
for m in reg.known_marks():
|
||||
m.detect(image, provenance=False)
|
||||
m.detect(image, provenance=True)
|
||||
assert single * 2 <= calls[0] + 2, f"perception did {single} sweeps vs {calls[0]} for two passes"
|
||||
|
||||
|
||||
class TestMarkKnowledgeIsOnTheRow:
|
||||
"""Registering a mark is ONE edit: the row carries everything about it.
|
||||
|
||||
Product family, label regime, the platform sentence and the metadata signals that
|
||||
confirm the vendor all used to live in separate hand-maintained tables across
|
||||
``watermark_registry``, ``identify`` and ``api``. That is how LibLibAI ended up
|
||||
registered but absent from the pill veto.
|
||||
"""
|
||||
|
||||
def test_identify_platform_table_is_derived_from_the_rows(self):
|
||||
from remove_ai_watermarks.identify import _VISIBLE_MARK_PLATFORM
|
||||
|
||||
assert {m.key: m.platform for m in reg.known_marks() if m.platform is not None} == _VISIBLE_MARK_PLATFORM
|
||||
|
||||
def test_the_platformless_marks_are_the_two_with_their_own_paths(self):
|
||||
"""Gemini has the higher-confidence sparkle path; the pill is too weak to
|
||||
attribute. Everything else must name a platform or `identify` reports none."""
|
||||
assert {m.key for m in reg.known_marks() if m.platform is None} == {"gemini", "jimeng_pill"}
|
||||
|
||||
def test_platform_scan_order_follows_the_registry(self):
|
||||
"""`identify` takes the FIRST platform match, so the order is load-bearing and
|
||||
must stay the registry's specificity order rather than a dict literal's."""
|
||||
from remove_ai_watermarks.identify import _VISIBLE_MARK_PLATFORM
|
||||
|
||||
assert list(_VISIBLE_MARK_PLATFORM) == [m.key for m in reg.known_marks() if m.platform is not None]
|
||||
|
||||
def test_every_tc260_mark_declares_the_aigc_signal(self):
|
||||
for mark in reg.known_marks():
|
||||
if mark.label_regime == "tc260" and mark.key != "jimeng_pill":
|
||||
assert "aigc" in mark.provenance_signals, mark.key
|
||||
|
||||
def test_only_gemini_claims_platform_tokens(self):
|
||||
by_token = {m.key for m in reg.known_marks() if m.provenance_platform_tokens}
|
||||
assert by_token == {"gemini"}
|
||||
|
||||
|
||||
class TestPillSuppressors:
|
||||
"""The pill veto is derived from the registry, not hand-listed.
|
||||
|
||||
The hand-written list drifted: LibLibAI was registered in the same commit as
|
||||
RunningHub and Baidu, both of which were added to the veto, and it was not. A
|
||||
derived set cannot be forgotten by the next registration.
|
||||
"""
|
||||
|
||||
def test_every_other_tc260_product_suppresses_the_pill(self):
|
||||
expected = {
|
||||
m.key
|
||||
for m in reg.known_marks()
|
||||
if m.label_regime == "tc260" and m.product != reg.get_mark("jimeng_pill").product
|
||||
}
|
||||
assert reg._pill_suppressors() == expected
|
||||
assert "liblib" in expected
|
||||
|
||||
def test_pill_dropped_on_liblib(self):
|
||||
assert not reg._keep_pill({"liblib"}, provenance=frozenset({"jimeng"}), footprint_flat=1.0)
|
||||
|
||||
def test_pill_dropped_on_liblib_even_with_the_jimeng_wordmark(self):
|
||||
"""The veto precedes the wordmark arm, so a co-firing LibLibAI wins.
|
||||
|
||||
This is the broader half of the change: it needs neither TC260 provenance nor
|
||||
a flat footprint, so it is reachable on more inputs than the metadata arm.
|
||||
"""
|
||||
assert not reg._keep_pill({"liblib", "jimeng"}, provenance=frozenset(), footprint_flat=1.0)
|
||||
|
||||
def test_pill_survives_gemini_and_samsung(self):
|
||||
"""Neither is a TC260 labeller, and neither can put "jimeng" into provenance,
|
||||
so neither may veto the arm it could not have enabled."""
|
||||
assert reg._keep_pill({"gemini", "jimeng"}, provenance=frozenset(), footprint_flat=1.0)
|
||||
assert reg._keep_pill({"samsung", "jimeng"}, provenance=frozenset(), footprint_flat=1.0)
|
||||
|
||||
def test_product_map_is_derived_from_the_rows(self):
|
||||
assert {m.key: m.product for m in reg.known_marks()} == reg._PRODUCT_OF
|
||||
assert reg._PRODUCT_OF["jimeng_pill"] == "jimeng" # the one shared product
|
||||
|
||||
|
||||
class TestProvenanceMaskThreading:
|
||||
"""Regression for the provenance-relaxed Gemini no-op (#1) and the false 'removed'
|
||||
label (#2). Before the fix, footprint_mask re-detected WITHOUT trust_provenance, the
|
||||
|
||||
Reference in New Issue
Block a user