Collapse the duplicated detection path and lift the image pipeline into the library

The visible-mark path had grown three copies of one ladder sweep, four
near-identical `detect` arms, and four hand-rolled `footprint_mask` overrides;
mark knowledge sat in five hand-maintained tables across three modules; and the
flagship `all`/`batch` pipeline existed only in cli.py, written twice with
divergent behavior.

Detection is now one measurement. `_ladder_best` replaces the three sweeps,
`_scan`/`_verdict` replace the four arms, and the winning box travels to the
mask on `TextMarkDetection.match_box` instead of being swept a second time.
`detect_both` returns the strict and relaxed verdicts from one scan, which
halves the arbiter's perception cost (260 -> 130 matchTemplate calls on a 2048²
image, verdicts identical field for field). A per-mark demotion goes in the new
`_post_gate` hook, never in a `detect` override -- an override is invisible to
the single-pass path, which is how the RunningHub and Yuanbao anchor gates
briefly stopped applying.

Everything about a mark is now one registry row: product, label regime, the
platform sentence `identify` reports, the metadata signals that confirm it, and
its TC260 producer codes. `identify._VISIBLE_MARK_PLATFORM`, the signal mapping
in `api.visible_provenance`, `_PRODUCT_OF` and the pill veto are derived from
those rows.

`api.remove_all` / `api.remove_batch` are the library form of the `all` and
`batch` commands; the CLI is a wrapper that owns console text and exit codes.
Progress is a `(stage, detail)` pair of stable tokens, so the CLI keys its
wording off structure rather than parsing the library's prose back.

Two intentional behavior changes, both verified against a recorded 811-image
sample of detector verdicts, removal-mask hashes, arbiter decisions and
`identify` reports:

  * A TC260 label now relaxes the vendor its `ContentProducer` names rather than
    ByteDance's pair on every China-AIGC image. 333 of 811 samples move; on 185
    of them the previously relaxed pair was simply the wrong vendor, and the
    mark actually present never reached the relaxed gate its own
    `provenance_ncc_factor` was calibrated for.
  * A confident LibLibAI detection suppresses the Jimeng pill, like every other
    TC260 product's mark. It was registered alongside RunningHub and Baidu, both
    of which were added to the hand-written veto list, and it was not. 1 sample
    moves, and it is exactly the co-firing case.

Nothing else in that record changes: detector verdicts, mask hashes and
`identify` verdicts are byte-identical, and all 200 calibration constants are
untouched.

Also: `aigc_label` and friends plus `extract_c2pa_info` are memoized on
(path, mtime_ns, size) -- size because this package rewrites in place; the
native TC260 container readers route on magic bytes instead of the file
extension, so a mislabeled AVI or FLV is no longer invisible; `identify` shares
one pixel decode between the DWT-DCT and visible stages (TrustMark keeps its own
Pillow decode, which is not substitutable); and the six `stabilize_*` video
wrappers collapse into one policy table.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

docs/cli.md

@@ -128,6 +128,12 @@ remove-ai-watermarks erase image.png \
 `erase` accepts `cv2`, `migan`, and `lama`. The corresponding optional extra
 must be installed for a learned backend.
 
+Two more knobs tune the fill. `--dilate N` (default 3) grows every box by `N`
+pixels before inpainting, which helps when a mark has a soft edge or a drop
+shadow just outside the box you measured; it applies to every backend because it
+shapes the mask. `--inpaint-method telea|ns` selects the classical algorithm and
+only affects the `cv2` backend.
+
 ## Strip AI metadata
 
 Inspect metadata:

docs/module-internals.md

@@ -253,9 +253,13 @@ takes the first stable result in specificity order (`sora`, `veo`, `seedance`,
 candidate. Removal also collects authoritative per-frame timestamps for the
 encoder, while identification omits that unused ffprobe pass.
 
-Every per-frame result is untrusted. The provider-specific stabilization
-wrappers share one recurrence implementation, while retaining separate visual
-floors and minimum-run policy. Provenance can relax a low-contrast run only
+Every per-frame result is untrusted. Each provider's floors, minimum-run policy,
+fill padding and mask style are one row in `VISIBLE_MARK_POLICIES`, and every mark
+enters the same `stabilize_localizations` entry point; the recurrence
+implementation underneath knows nothing about providers. That policy row also
+carries `accepts_provenance`, which forces `provenance=False` for Hailuo and Kling
+— they have no metadata that could confirm them, and the guarantee used to be
+structural (their wrappers took no `provenance` parameter at all). Provenance can relax a low-contrast run only
 after recurring visual evidence exists. Sora transition frames follow the
 nearest confirmed moving position only with Sora provenance. Veo, Seedance,
 Dola, Hailuo, and Kling additionally require candidates to remain anchored to
@@ -355,6 +359,43 @@ metadata extraction from verdict logic:
 - `identify` preserves the path-based API and adds the optional registered
   visible-mark and open invisible-watermark decoders after extraction.
 
+The DWT-DCT detector and the visible-mark stage share a single decode of the
+source, held by
+a per-call `_SharedDecode`. It exposes two accessors because the two arms need
+opposite failure handling: the visible arm swallows a decode failure (no cv2, no
+visible marks, metadata verdict untouched), while the invisible arm re-raises it
+so `has_invisible_target` reaches its documented fail-safe `True`. Swallowing it
+there would skip a diffusion scrub on a file that used to get one. TrustMark
+deliberately keeps its own Pillow decode: cv2 and Pillow disagree on EXIF
+orientation and on 16-bit PNG, so substituting one for the other is not
+behavior-preserving.
+
+The metadata probes (`aigc_label`, `xai_signature`, `iptc_ai_system`,
+`huggingface_job`, `samsung_genai`) and `extract_c2pa_info` are memoized on
+`(path, mtime_ns, size)`. One `identify` reaches each of them twice, and each
+re-walks the container or re-runs the manifest reader. Size is in the key as well
+as mtime because this package rewrites files in place, and an in-place rewrite
+can land inside one mtime tick. The C2PA key additionally carries the
+reader-availability flag: with the official reader the manifest comes back as a
+store and without it from the PNG chunk parser, so the answer depends on process
+state and not on the file alone.
+
+Native-container TC260 readers (`isobmff`, `ebml`, `riff`, `flv`) all run, in that
+order, on every file. Each self-gates on its own magic bytes after a 4-12 byte
+read, so gating the AVI and FLV ones on the file extension as well was redundant
+and made a correctly formatted container served under the wrong name invisible.
+WebP is the one input class the now-unconditional RIFF reader newly touches; its
+`AVI ` form check is what rejects it.
+
+`api._SourceEvidence` extracts that metadata once per `remove_all` call and serves
+both the visible pass (which vendor is confirmed) and the scrub gate (is there an
+invisible target). It is per-call, never module-level: `batch` may write its
+output over its input, and a holder that outlived one call would answer the scrub
+gate from pre-write evidence. In `batch` each stage builds its own holder after
+any write that precedes it, for the same reason. Every accessor fails safe the way
+the function it replaces does — no provenance means no relaxation, and an unknown
+invisible target means scrub rather than skip.
+
 The `detect` extra composes the shared `pixels` runtime with PyWavelets. Its
 in-tree [`dwt_dct.py`](../src/remove_ai_watermarks/dwt_dct.py) decoder preserves
 the upstream matrix algorithm without installing Torch or non-headless OpenCV.
@@ -397,6 +438,30 @@ The Jimeng pill has an additional decision gate because its visual detector is
 weaker than the other registered marks. Keep that policy in the registry, not
 inside unrelated detector engines.
 
+Everything about a mark is one registry row: its product family, its label regime,
+the platform sentence `identify` reports for it, and the metadata signals that
+confirm its vendor. `identify._VISIBLE_MARK_PLATFORM` and the signal mapping in
+`api.visible_provenance` are derived from those rows rather than hand-maintained
+beside them, so registering a mark is one edit. Two marks carry no platform of
+their own: the Gemini sparkle has its own higher-confidence path, and the
+capture-less pill is too weak to attribute.
+
+The set of marks that veto the pill is DERIVED from the registry rows: every mark
+under the same label regime (`tc260`) belonging to a different product. It used to
+be a hand-written list of keys, and that list drifted -- LibLibAI was registered
+alongside RunningHub and Baidu, both of which were added to it, and LibLibAI was
+not, so a confident LibLibAI detection did not suppress the pill the way its two
+siblings did. Marks outside the TC260 regime (Gemini, Samsung) are deliberately
+not vetoers: neither can put `jimeng` into `provenance`, so neither can enable the
+arm it would be vetoing.
+
+A TC260 label relaxes the vendor its `ContentProducer` names, resolved through
+`KnownMark.tc260_producer_codes`. The label itself is vendor-agnostic, so this used to
+relax ByteDance's two products on every China-AIGC image -- which both risked a
+false fill on an image carrying some other vendor's mark and denied that vendor's
+own mark the relaxed gate its `provenance_ncc_factor` was calibrated for. An
+absent or unmapped producer still falls back to the ByteDance pair.
+
 `remove_auto_marks` removes every selected mark, not only the strongest one.
 This matters for images that carry marks in more than one corner.
 
@@ -440,7 +505,23 @@ be represented by the shared base:
 
 The detector and removal mask must use compatible geometry. A detector that
 fires while producing an empty or misplaced mask is a removal failure even if
-the detection test passes.
+the detection test passes. That parity is now structural rather than a
+convention: the three continuous front ends share one ladder sweep
+(`_ladder_best`), and the winning box travels to the mask on
+`TextMarkDetection.match_box` instead of being swept a second time.
+
+Detection is split into a trust-level-blind `_scan` and a `_verdict` that applies
+the threshold. `detect_both` returns the strict and relaxed verdicts from one
+scan, which is what the arbiter's perception stage calls. A per-mark demotion
+belongs in the `_post_gate` hook, never in a `detect` override: an override is
+invisible to the single-pass path, and the RunningHub and Yuanbao anchor gates
+were briefly skipped there for exactly that reason.
+
+A mark whose removable footprint differs from what the detector localizes
+overrides `_footprint_rect` (which policy) and `_extend_match_box` (how far the
+box grows), not the whole `footprint_mask`. Baidu extends right to the corner tag
+and LibLibAI extends left to the triangle logo; both inherit every guard around
+that arithmetic.
 
 Yuanbao uses the polarity-independent `contrast` front end because its standard
 two-line mark can be light on dark scenes or dark on light scenes. Its detector

docs/python-api.md

@@ -77,6 +77,55 @@ image = cv2.imread("input.png")
 result, removed = raiw.remove_visible(image, backend="cv2")
 ```
 
+## Run the full pipeline
+
+`remove_all` is the library form of the `all` command: visible marks, then the
+invisible watermark, then AI metadata. Stages are chained through a file in the
+system temp directory, so a partial result never appears at the output path.
+
+```python
+import remove_ai_watermarks as raiw
+
+result = raiw.remove_all("input.png", "clean.png")   # -> RemoveAllResult
+print(result.output)          # the path written
+print(result.visible_label)   # the marks removed, or None
+print(result.invisible)       # "removed" | "no-signal" | "unavailable"
+```
+
+`invisible` is the field to check. `"unavailable"` means the GPU extra is not
+installed, so the output *looks* processed but still carries the watermark;
+`"no-signal"` means the scrub was deliberately skipped because nothing was
+locally detectable, which is a successful run.
+
+Pass `InvisibleOptions` to tune the diffusion stage, and `engine` to reuse one
+loaded model across many calls:
+
+```python
+from remove_ai_watermarks import InvisibleOptions
+
+raiw.remove_all(
+    "input.png",
+    "clean.png",
+    invisible=InvisibleOptions(strength=0.35, force=True),
+    progress=print,
+)
+```
+
+If AI metadata survives the strip, `remove_all` raises `MetadataStripIncomplete`
+**before** writing anything: an AI-readable output is worse than no output.
+
+`remove_batch` runs one mode over a directory and never lets a single bad file
+end the run:
+
+```python
+summary = raiw.remove_batch("in_dir", "out_dir", mode="visible")   # -> BatchSummary
+print(summary.processed, summary.failed, summary.errors)
+print(summary.invisible_unavailable)   # outputs that still carry the watermark
+```
+
+`mode` is `all`, `visible`, `invisible`, or `metadata`. Pass a constructed
+`InvisibleEngine` as `engine` to load the model once for the whole directory.
+
 ## Inspect provenance
 
 The default installation evaluates file metadata. Add `visible`, `detect`, or
@@ -151,9 +200,22 @@ describe the collector rather than the source file. Pass a C2PA manifest-store
 dictionary in `record["c2pa_store"]`, or through the explicit
 `c2pa_manifest_store` argument.
 
-`identify_from_evidence` does not reopen the source file. It evaluates metadata
-only; registered visible marks and pixel-backed invisible watermarks remain in
-the path-based `identify` call.
+`identify_from_evidence` does not reopen the source file by default: it evaluates
+metadata only, and registered visible marks and pixel-backed invisible watermarks
+remain in the path-based `identify` call.
+
+Pass `image_path` together with `check_visible` or `check_invisible` to add those
+pixel detectors on top of the SAME evidence. That is how a caller asking one file
+two provenance questions — which vendor is confirmed, and is there an invisible
+target — pays for the metadata extraction once:
+
+```python
+from remove_ai_watermarks.identify import extract_provenance_evidence, identify_from_evidence
+
+evidence = extract_provenance_evidence(source)
+metadata_only = identify_from_evidence(evidence)
+with_pixels = identify_from_evidence(evidence, image_path=source, check_invisible=True)
+```
 
 ## Strip metadata
 

docs/watermarking-landscape.md

@@ -36,7 +36,7 @@ AI generator values while retaining unrelated camera and editor EXIF. The
 shared `xai_signature_pair` helper is the single source of truth for the
 pair. On the ISOBMFF path, `blank_ai_exif_tokens` provides the corresponding
 in-place scrub for supported EXIF values, TC260 AIGC blocks, and the xAI pair.
-- **China TC260 AIGC label (caught by `AIGC_MARKERS` / `metadata.aigc_label`, surfaced by `identify` as the `aigc` signal):** China-served generators embed an XMP `<TC260:AIGC>{"Label":"1","ContentProducer":...}` block — China's mandatory AI-content labeling (TC260 namespace `tc260.org.cn/ns/AIGC`).
+- **China TC260 AIGC label (caught by `AIGC_MARKERS` / `metadata.aigc_label`, surfaced by `identify` as the `aigc` signal):** China-served generators embed an XMP `<TC260:AIGC>{"Label":"1","ContentProducer":...}` block — China's mandatory AI-content labeling (TC260 namespace `tc260.org.cn/ns/AIGC`). The label says only "this is AI", but its `ContentProducer` names the signing entity — `001` + `1` + an 18-char Unified Social Credit Code + a 5-digit product suffix, normalized by `metadata.uscc_of`, or for a few generators a bare product name. `KnownMark.tc260_producer_codes` maps the codes settled per vendor by `scripts/vendor_cohort_harvest.py` to registry mark keys, so an AIGC image relaxes the detector of the vendor it actually carries rather than a guessed pair; an unmapped or absent producer falls back to ByteDance's two products. A code identifies a legal entity, not necessarily one brand, so a hosting or aggregating platform that signs for several apps is a recall bet rather than a proof.
 
 **Doubao** (ByteDance) uses it (verified on a public issue sample; `ContentProducer` `001191110102MACQD9K64010000`, no C2PA/SynthID/imwatermark — the XMP block is the only signal; GitHub attachment upload did NOT strip it). The same standard is mandatory for Jimeng/Kling/Qwen/Ernie etc., so the one marker covers the whole China-AIGC-labeled ecosystem. `aigc_label` reads **four image serializations** through a shared `_parse` helper: the HTML-entity-encoded XMP `TC260:AIGC` block in **either RDF form** — the nested element `<TC260:AIGC>{...}</TC260:AIGC>` (Doubao) or the attribute `TC260:AIGC="{...}"` (**PicWish**, `ContentProducer="picwish"`, verified on compatible samples) — via a container-agnostic raw-byte scan (any JSON object accepted), a raw-JSON PNG `AIGC` tEXt chunk (Doubao also writes the label this way, no namespaced marker at all — confirmed on compatible samples, `ContentProducer="doubao"`), a bare raw-JSON `{"AIGC":{...}}` object embedded in **JPEG EXIF (UserComment)** by some China-served generators, brace-matched from the scan head with `json.JSONDecoder().raw_decode` (no namespaced marker, no PNG chunk — confirmed on compatible samples, `ContentProducer="001191440300708461136T1308L"`), **and** a bare `AIGC{...}` blob (the label glued straight to its JSON, no `"AIGC":` key wrapper) embedded in a **JPEG APP segment near the JFIF header** — confirmed on compatible samples. The two raw-JSON forms are scanned in one loop (`'"AIGC"'` then `AIGC{`) that **falls through on a non-TC260 / undecodable hit instead of returning** — a quoted `"AIGC"` can appear later in an XMP packet while the real label is a bare `AIGC{...}` earlier in the file, so an unconditional early return on the quoted form would shadow the bare form (the exact bug behind the 06-10 misses). Native MP4/MOV is a fifth serialization: TC260-PG-20257A stores an `AIGC` key in `moov.udta.meta.keys` and the raw JSON in the matching `ilst` item. The seeking parser reaches a tail `moov` without reading `mdat`; removal replaces the key with `free` and blanks the validated value at the same length so every box size and stream offset stays fixed. All generic forms are gated on at least one TC260 field (`TC260_AIGC_FIELDS`) so a generic `AIGC` key cannot false-positive; the namespaced XMP element is unambiguous and needs no gate. `TC260_AIGC_FIELDS` covers **two schemas**: the producer-side one (`Label` / `ContentProducer` / `ProduceID` / `ContentPropagator` / `PropagateID`, Doubao and most China gens) and the **service-provider** one (`ServiceProvider` / `ServiceUser`, plus generic `Time` / `ContentId` which are NOT gated on) — **Tencent Cloud's** AIGC variant (`ServiceProvider` = `腾讯云`), embedded in **EXIF `ImageDescription`**, verified on compatible samples. In `identify`, `aigc` fires on the parsed label **or** the `AIGC_MARKERS` byte scan (the latter preserves the laundering-tell case where the JSON payload is truncated).