The 'no signal' branch of the visible no-mark path claimed 'No AI provenance
signal found either', which reads as 'the image is clean'. A missing metadata
proxy is not proof an invisible pixel watermark (SynthID) is absent: it cannot
be detected once metadata is gone and may have been stripped upstream. The
message now preserves that uncertainty and routes to both 'all' (regenerate
pixels) and 'erase'. Regression-guarded by the SynthID/all asserts in
test_cli.py. CLAUDE.md visible-command note updated to match.
Also adds a 'Scope and non-goals' section (CLAUDE.md + README): removing
AI-provenance marks on the user's own content is in scope; stripping
stock/paid-content watermarks (Shutterstock/Getty/iStock, classifieds) is out
of scope by principle, not by difficulty.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When `visible --mark auto` (or an explicit `--mark` with detection on) found
no registered mark, it exited 0 without writing output -- which a wrapping
service reads as success and re-serves the unchanged input. ~74% of real
uploads carry no registered visible mark, so this was the dominant "it didn't
work" / NPS score-0 failure mode.
Now it runs a cheap metadata-only identify, prints actionable guidance (route
to `all` for an invisible/metadata mark, or `erase` for an arbitrary logo),
writes no output file, and exits EXIT_NO_VISIBLE_MARK (2) -- distinct from
success (0) and a hard error (1) so the caller can surface the message.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Step 2 (invisible/SynthID) was skipped with a quiet inline warning and the
run still exited 0, so a missing [gpu] extra was mistaken for a clean result
(recurring #14/#47). Add a prominent end-of-run banner and a non-zero exit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Nine findings from a high-effort project-wide review, fixed and verified
(571 passed, ruff/pyright clean):
Correctness:
- all/batch now remove Doubao/Jimeng/Samsung visible text marks: the visible
step routes through the registry (new cli._remove_visible_auto) instead of a
hardcoded GeminiEngine, so they no longer leave the wordmark intact.
- batch always reads the original source (dropped the out_path-reuse that
re-processed already-cleaned outputs on a re-run).
- img2img_runner only retries the diffusion call on the deprecated-callback
TypeError; any other TypeError now propagates instead of double-running.
- gemini detect/remove and the reverse-alpha engines normalize channels via a
new image_io.to_bgr, fixing a grayscale/BGRA crash in the FP-gate path.
- _png_late_metadata advances its cursor by the clamped length, so a malformed
chunk length no longer aborts the late AI-label scan.
Cleanup / efficiency:
- Consolidate the ~90%-identical Doubao/Jimeng/Samsung engines into a shared
config-driven _text_mark_engine.TextMarkEngine base; each engine is now a thin
subclass (TextMarkConfig + test shims). Behavior is byte-exact (the three
engine test suites pass unchanged). Registry adapters collapse to one
_text_mark(...) row each. Gemini stays a separate engine.
- scan_head is memoized per (path, size, mtime), so identify() reads the file
head once instead of ~8 times.
- invisible_engine post-processing decodes/encodes the output once (chained in
memory) instead of 2-4 times across stages.
- Remove the orphaned get_model_id_for_profile (+ CONTROLNET_PROFILE); derive
the --strength help from the strength constants (strength_default_help) so it
cannot drift; share the --pipeline/--strength click options; simplify the
retired --auto resolver.
Net -835 lines. Tests added for the registry-routed visible pass, to_bgr,
the polish/model/guidance wiring, and strength_default_help. CLAUDE.md updated
for the new base module, the engine/registry changes, image_io.to_bgr, and the
scan_head cache.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Overhaul the diffusion-removal surface around a single robust default and a
complete, consistent CLI.
Pipeline + strength:
- controlnet is now the DEFAULT pipeline (CLI --pipeline + both engine ctors).
With the certified higher strength it clears both photoreal and flat-graphic
content, whereas plain SDXL left SynthID on flat graphics.
- Rename the plain-SDXL profile default -> sdxl; "default" stays as a back-compat
alias (normalize_profile + a click callback that warns).
- Unify the strength ladder: resolve_strength applies ONE vendor-adaptive ladder
(the certified controlnet floors OpenAI 0.20 / Google 0.30 / unknown 0.30) to
both pipelines. sdxl is the weaker remover on its own hard case (flat fills),
so the certified floor is the right floor for it too.
CLI completeness:
- Add --model (HF model id) to invisible + batch (was only on all) and
--guidance-scale (CFG) to all three diffusion commands; both were library
knobs the CLI did not expose.
- Flip --adaptive-polish to ON by default (it self-gates to a no-op where there
is no detail deficit, so default-on is safe).
- Share --pipeline / --strength / --model / --guidance-scale as single
decorators so invisible/all/batch keep an identical surface; the --strength
help is derived from the strength constants (strength_default_help) so it can
never drift from the ladder.
Removals:
- Delete the auto_config content-detection planner + its YuNet/DBNet assets
(~2.6 MB): with controlnet always the pipeline and the polish self-gating, the
face/text/edge detection no longer changed behavior. --auto is now a deprecated
no-op that only warns (the polish it enabled is the default).
Docs (README, CLAUDE.md, docs/synthid.md) updated throughout; added an
InvisibleEngine Python API example. Tests cover the alias warnings, the
polish default, and the --model/--guidance-scale wiring.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The face-restore family was removed in 20d7eda, but the auto_config
module docstring still claimed "PhotoMaker face restoration is enabled
when a face is present" and the --auto help text (CLI + README example)
listed "face restore" as something --auto picks. A detected face now
only routes to the controlnet pipeline (canny preserves face STRUCTURE,
not identity); there is no identity restoration. Comments/docstrings/help
only, no code behavior change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Empirical conclusion from the 2026-06-04 - 2026-06-08 Modal cert sweeps:
every face-restore approach we built (GFPGAN-on-cleaned, PhotoMaker-V2,
InstantID txt2img, InstantID img2img-on-cleaned at three parameter
settings) regenerates the face via SDXL diffusion rather than preserves
it. Output face pixels are diffusion-fresh, so the regenerated face
inherits SDXL "clean skin" aesthetic and loses original identity
precision -- it looks MORE AI-generated than the cleaned image, not
less. The cleaned image from the main controlnet 0.20 removal pass is
the least-AI face state we can reach without re-introducing SynthID.
Nothing in the restore family achieves the actual goal (preserve the
original person's face). Keeping them around as opt-in invites users to
ship something that defeats the point. Removing entirely.
Library changes:
- Deleted src/remove_ai_watermarks/instantid_restore.py
- Deleted src/remove_ai_watermarks/photomaker_restore.py
- Deleted tests/test_instantid_restore.py
- Deleted tests/test_photomaker_restore.py
- Removed `instantid` and `photomaker` extras from pyproject.toml
- Removed `[tool.hatch.metadata] allow-direct-references = true` (was
only needed for the photomaker git+ URL)
- InvisibleEngine.remove_watermark: dropped `restore_faces` +
`restore_faces_method` params, removed both `_restore_faces_instantid`
and `_restore_faces_photomaker` private methods, removed dispatch
- CLI: dropped `_restore_faces_options` decorator, all four cmd_*
signatures lose `restore_faces` + `restore_faces_method`, kwarg passes
to remove_watermark dropped
- _apply_auto: dropped `restore_faces` from tuple shape (was unused after
the engine no longer takes it)
- auto_config.AutoConfig: dropped `restore_faces` field; `plan()` no
longer sets it; `reason` no longer mentions it
- Tests updated accordingly (test_auto_config.TestReason no longer asserts
"face-restore on" in the reason string)
Docs updated:
- CLAUDE.md: removed the photomaker extras bullet, the Face restore
trade-off bullet, the instantid_restore.py + photomaker_restore.py
module bullets; replaced restore mentions in watermark_remover and
controlnet bullets and prod recipe with the empirical conclusion
- README.md: removed both `--restore-faces` callouts and the install
snippet; the feature bullet and auto-mode comment updated
- docs/synthid-robust-identity-research.md: added Status-retired notice
at the top pointing at the 2026-06-08 followup
raiw-app:
- modal_cert.py: dropped `--restore-faces` flag entirely; sweep() no
longer takes restore_faces; pinned _LIB_SPEC to `[gpu]` extras (no
`photomaker` / `instantid` extras), points at main
ruff + strict pyright clean; 569 tests pass; 18 restore-specific tests
gone.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Empirical conclusion from the 2026-06-04 - 2026-06-08 cert sweeps:
every shipped face-restore method (GFPGAN-on-cleaned, PhotoMaker-V2,
InstantID txt2img, InstantID img2img-on-cleaned at three parameter
settings) regenerates the face from an ArcFace embedding via SDXL
diffusion. Output face pixels are diffusion-fresh, which makes the
regenerated face look MORE AI-generated than the cleaned image (gloss,
symmetric pores, SDXL "clean skin" aesthetic) regardless of license.
The cleaned image from the main controlnet 0.20 removal pass is the
LEAST-AI state we can reach without re-introducing SynthID; any restore
on top trades original-look for embedding-driven regeneration. The
fundamental issue is structural: ArcFace encodes "general look" at 512
dimensions, SDXL decodes that into pixels with the inherent SDXL
aesthetic. Stronger identity push (higher strength + IP-Adapter scale)
makes the face closer to the embedding but more AI-looking; weaker push
leaves identity to drift further. No parameter setting recovers original
identity AND looks less AI than cleaned.
Production conclusion: do not ship `--restore-faces` in any monetized
deployment. The extras (`instantid`, `photomaker`) stay in the library
for research / personal use where users explicitly want regeneration.
Documented at every entry point:
- CLAUDE.md: new "Face restore trade-off" bullet + every restore mention
rewritten to "REGENERATES, does NOT recover"; controlnet bullet updated
- README.md: feature bullet + callout + secondary mention all updated
- docs/synthid-robust-identity-research-2026-06-08.md: appended
"Empirical follow-up" section documenting the InstantID sweep phases
(Phase 1 txt2img v1/v2/v3, Phase 2 img2img defaults + stronger params)
- docs/controlnet-removal-pipeline-research.md: updated restore-faces
bullet to reflect the empirical conclusion
- CLI help: `_restore_faces_options` docstring + `--restore-faces` /
`--restore-faces-method` help text all updated
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per the 2026-06-08 deep-research synthesis (docs/synthid-robust-identity-
research-2026-06-08.md), the entire ArcFace-class identity-adapter ecosystem
for SDXL is blocked from commercial use by InsightFace's non-commercial model
packs (antelopev2 / buffalo_l). No commercial-safe ArcFace-grade identity
stack exists today. The user explicitly opted into shipping a non-commercial
restore path (research / personal use; raiw.cc must NOT install the extra).
Architectural choice: InstantID over PhotoMaker-V2 as the default.
- PhotoMaker-V2 (CLIP+ArcFace dual encoder, txt2img only): documented upstream
identity drift on Asian male faces, visually confirmed in our cert sweep
(tatsunari rendered as a generic woman; group photo collapsed into a
patchwork).
- InstantID (ArcFace cross-attention + landmark ControlNet): semantic
identity branch + spatial weak landmark control, decoupled. Per InstantID
paper (arXiv:2401.07519) and the research report, stronger identity fidelity
on single portraits. Critically: NO original face pixels enter the diffusion
(ArcFace embedding is semantic, landmark stick figure is pure geometry), so
SynthID is not transported.
Implementation:
- New `src/remove_ai_watermarks/instantid_restore.py` mirrors the
`photomaker_restore.py` shape (lazy singletons for pipeline + FaceAnalysis,
per-face crop + _composite_faces from photomaker_restore). Loads the
InstantID community pipeline via `DiffusionPipeline.from_pretrained(
custom_pipeline="pipeline_stable_diffusion_xl_instantid")` -- no upstream
Python package needed; diffusers fetches the file from its community
examples.
- New `instantid` extra in pyproject (insightface + onnxruntime +
huggingface-hub). NON-COMMERCIAL block in the comment explains why.
- CLI: `--restore-faces-method [instantid|photomaker]`, default `instantid`.
Both methods explicitly labeled NON-COMMERCIAL in the help text.
- Engine: dispatch on `restore_faces_method` to either
`_restore_faces_instantid` or `_restore_faces_photomaker`.
- 9 control-flow tests for InstantID without model download (mirror the
photomaker_restore.py test pattern + draw_kps helper checks). 587/587 pass.
Diffusers-0.38 compat verified by upstream code inspection: the InstantID
pipeline inherits from `StableDiffusionXLControlNetPipeline`, uses only
public diffusers APIs (`encode_prompt`, `prepare_image`, `prepare_latents`,
`get_guidance_scale_embedding`), uses legacy attention processor API which
diffusers preserves for backward compat. No PhotoMaker-V1-style internal
text_encoder access. End-to-end execution will be validated by the Modal
cert sweep in the next step.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Visual review of the GFPGAN-on-cleaned output (9-face grid, 1448x1086) showed it
only polished the already-drifted face without restoring identity — useless for the
"restore who is in the photo" intent. Dropping it.
The shipped restore path is now PhotoMaker-V2, which delivers true identity-from-
embedding face regeneration via a CLIP+ArcFace dual encoder. The ArcFace branch
pulls InsightFace antelopev2/buffalo_l model packs at runtime, which InsightFace
releases under a research-only license, so the whole extra is **NON-COMMERCIAL**.
raiw.cc and any monetized deployment must NOT install the `photomaker` extra.
This is called out at every entry point: CLI flag help, module docstring,
pyproject extra block, CLAUDE.md extras bullet, README install snippet.
Changes:
- Deleted `src/remove_ai_watermarks/face_restore.py` and its tests.
- Deleted the `restore` extra (gfpgan/facexlib/basicsr + scipy<1.18 / numba<0.60
pins) and the basicsr setuptools<69 build pin from pyproject.toml.
- Restored `src/remove_ai_watermarks/photomaker_restore.py` (V2 this time:
`TencentARC/PhotoMaker-V2`, `photomaker-v2.bin`, no `pm_version='v1'` override).
- Restored the `photomaker` extra in pyproject with all the upstream-compat
pins (einops, peft, onnxruntime, insightface) and the `allow-direct-references`
hatch metadata block.
- `InvisibleEngine` swapped `_restore_faces` -> `_restore_faces_photomaker`;
`--restore-faces-method` removed (only one method, no choice).
- CLI flag help, CLAUDE.md, README, docs/synthid.md, and
docs/controlnet-removal-pipeline-research.md all updated.
- docs/synthid-robust-identity-research.md status notice rewritten to list both
abandoned commercial-safe attempts (V1 + GFPGAN-on-cleaned) and the
non-commercial trade-off we accepted.
ruff + strict pyright(src/) clean; 578 tests pass (the 9 GFPGAN tests are gone,
the 11 PhotoMaker tests stay green).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
After 7 cascading upstream-compat fixes (insightface dep, peft dep, pm_version,
device, etc.), the PhotoMaker V1 cert sweep still hit a CFG batch-dim mismatch
inside the denoising loop. The upstream PhotoMaker `pipeline.py` is forked from
diffusers v0.29.1 and our env runs 0.38; SDXL prompt-encoder handling changed
significantly between those versions, so making PhotoMaker work end-to-end
needs a proper fork or a diffusers downgrade — both expensive. Not worth
shipping today.
Pivot: restore `face_restore.py` (GFPGAN) with a single-line fix that makes it
SynthID-safe by construction. The previous design ran GFPGAN.enhance on the
ORIGINAL watermarked image and was oracle-confirmed to re-add SynthID via the
weight-0.5 pixel blend. The fix is to run GFPGAN on the diffusion-CLEANED
image — whatever pixels GFPGAN derives from are already SynthID-free, so the
partial blend cannot transport the watermark. Identity fidelity is lower than
a true identity-as-embedding stack would deliver, but it ships and works.
Changes:
- `src/remove_ai_watermarks/face_restore.py` restored from pre-wipe state with
one line changed: `restorer.enhance(cleaned_bgr, ...)` instead of
`restorer.enhance(original_bgr, ...)`. `original_bgr` is kept as an unused
positional argument for API stability.
- `src/remove_ai_watermarks/photomaker_restore.py` and its tests REMOVED. The
research note (`docs/synthid-robust-identity-research.md`) keeps a "status
notice" documenting why PhotoMaker is parked for now and what the path back
in would look like.
- `pyproject.toml` `restore` extra restored (gfpgan/facexlib/basicsr +
scipy<1.18 + numba<0.60 pins + the basicsr setuptools<69 build pin), plus
`photomaker` extra (with its einops/insightface/peft pile) and the
`[tool.hatch.metadata] allow-direct-references = true` block REMOVED.
- `InvisibleEngine._restore_faces_photomaker` removed; `_restore_faces`
restored. The `--restore-faces` CLI flag and its plumbing through cmd_*
signatures are unchanged.
- CLAUDE.md, README.md, docs/synthid.md, docs/controlnet-removal-pipeline-
research.md updated to describe the shipped GFPGAN-on-cleaned design and to
reference PhotoMaker only as the parked alternative.
ruff + strict pyright(src/) clean; 578 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The GFPGAN `restore` extra and its `face_restore.py` module are gone. They were
oracle-confirmed to re-introduce SynthID by blending watermarked original face
pixels at fidelity weight 0.5 (clean A/B: gemini_3 controlnet 0.20 detected WITH
GFPGAN, clean WITHOUT). Keeping them as the default restore method was a footgun
for the removal pipeline. PhotoMaker-V2 (added in the previous commit) is the
single shipped restore path now -- identity-as-embedding, SynthID-safe by
construction.
Removed:
- src/remove_ai_watermarks/face_restore.py + tests/test_face_restore.py
- pyproject.toml `restore` extra (gfpgan/facexlib/basicsr + scipy/numba pins)
- pyproject.toml `[tool.uv.extra-build-dependencies] basicsr = [...]` build pin
- CLI: `--restore-faces-method` and `--restore-faces-weight` (no method choice
to make, no GFPGAN weight knob to expose)
- InvisibleEngine._restore_faces method (only _restore_faces_photomaker remains)
- All restore-faces-method / restore-faces-weight threading through cmd_*
signatures and _process_batch_image
Kept:
- `--restore-faces / --no-restore-faces`: now binds to PhotoMaker-V2.
- All adopted oracle findings about GFPGAN re-introducing SynthID (kept in the
research docs as historical context that explains why the path was removed).
Docs updated: CLAUDE.md (restore extras bullet collapsed to photomaker, removed
face_restore Key-modules bullet, several inline GFPGAN refs scrubbed), README.md
(face-identity callout + install section now point to the photomaker extra),
docs/synthid.md 5.5 (net recipe), docs/controlnet-removal-pipeline-research.md
(recommendations).
ruff + strict pyright (src/) clean; 578 tests pass (the 9 GFPGAN tests are gone,
the 9 PhotoMaker tests stay green).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the second face-restore mechanism, selectable via the new CLI option
`--restore-faces-method=photomaker`. Unlike the existing GFPGAN path (which runs on
the watermarked ORIGINAL and was oracle-confirmed to re-introduce SynthID by partial
pixel blending), PhotoMaker carries identity in a SynthID-invariant OpenCLIP
embedding and regenerates fresh face pixels conditioned on it — the pixels in the
output are diffusion-fresh, so the watermark cannot be transported.
The load-bearing assumption (embedding invariance to SynthID-magnitude pixel noise)
was empirically validated in the prior commit (smoke test): cosine drift 0.002
under a ±2 LSB low-freq carrier, an order of magnitude less than JPEG90 drift
which SynthID survives at >=99% TPR.
End-to-end commercial-safe:
- PhotoMaker-V2 weights: Apache-2.0 (TencentARC)
- ID encoder: OpenCLIP-ViT-H/14 (MIT)
- SDXL base: shared with the main pipeline
- NO InsightFace (the non-commercial blocker for IP-Adapter FaceID / InstantID /
PuLID / Arc2Face)
Two-pass architecture (PhotoMaker has no ControlNetImg2img class in diffusers):
1) main controlnet/default removal pass cleans SynthID + drifts faces
2) PhotoMaker txt2img regenerates each face from its embedding, feather-composited
back into the cleaned image
New module `photomaker_restore.py` mirrors `face_restore.py`: lazy pipeline
singleton (double-checked lock), `is_available()` gate, pure `_face_crop_square` and
`_composite_faces` helpers, all unit-tested without the model (9 new tests). New
`InvisibleEngine._restore_faces_photomaker` runs after the diffusion pass, mirroring
`_restore_faces`. CLI flag `--restore-faces-method=[gfpgan|photomaker]` threaded
through `cmd_invisible`/`cmd_all`/`cmd_batch` + `_process_batch_image`.
New optional `photomaker` extra (Apache-2.0 + Apache-2.0/MIT deps, no basicsr).
`[tool.hatch.metadata] allow-direct-references = true` is required because the
upstream PhotoMaker package lives only on GitHub.
The next step (separate work) is oracle validation: run a 6-image cert sweep
through the new pipeline (default/controlnet at the certified strength +
--restore-faces-method=photomaker) and confirm SynthID stays clean while face
identity is recovered. The required infrastructure (`raiw-app/modal_cert.py`) is
already in place.
ruff + strict pyright(src/) clean; 586 tests pass (+ 9 new in
tests/test_photomaker_restore.py).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three content-quality features for the invisible/all/batch pipeline.
DBNet text detector (auto_config): replace the MSER text heuristic with
PP-OCRv3 differentiable-binarization via cv2.dnn.TextDetectionModel_DB,
using a bundled 2.4 MB Apache-2.0 model (en/cn detection nets are
byte-identical, so it ships language-neutral). cv2.dnn is core OpenCV, so
no new pip dep. MSER stays as the fallback when the model can't load.
Validated on real images: matches MSER everywhere and additionally catches
the Doubao CJK mark MSER missed; routing decisions unchanged otherwise.
Real-ESRGAN upscaler (new upscaler.py, esrgan extra): optional
pre-diffusion super-resolution for the min-resolution floor upscale, loaded
via spandrel (MIT, no basicsr) with BSD-3-Clause weights downloaded on
first use. New --upscaler {lanczos,esrgan} on invisible/all/batch; default
stays lanczos and the engine falls back to lanczos when the extra is absent
or the model errors (never breaks removal). It is a manual opt-in knob (the
auto plan never selects it) -- as a generic GAN it sharpens photo/texture
content strongly but can degrade faces (the diffusion pass regenerates
them) and thin text, documented accordingly.
batch --auto: wire the content-adaptive --auto (+ --adaptive-polish) into
cmd_batch. The plan is recomputed per image and the invisible engine is
cached per resolved pipeline (default/controlnet), so a mixed directory
builds at most one engine of each kind. Verified end-to-end: 3 mixed
images routed correctly with only 2 pipeline loads (controlnet reused).
ruff + strict pyright(src/) clean; 558 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Brings in commit 5cf68a6 (single C2PA_AI_VENDORS registry, erase_lama
grayscale/BGRA support, batch device-cache clearing + --controlnet-scale,
uv publish via OIDC, hatchling pin <1.31). Auto-merged with no conflicts;
ruff/pytest(544)/pyright all clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The fixed mild auto polish (unsharp 0.5 / grain 2.0) under-corrected soft
photo/face output (gemini_3 stayed at lap-var 84 vs its 592 original) and its
grain speckled small text. Replace it with humanizer.adaptive_polish: target the
input's Laplacian variance with a capped unsharp scaled to the deficit + edge-
masked grain (smooth regions only), calibrated by a short sigma search. Self-
limiting on text/graphics -- already high-frequency, so almost no polish lands
and text edges are masked out. Validated on the spaces corpus (gemini_3 84 -> 334
end-to-end; openai_1 text near-untouched).
Interface: every --auto decision is now independently overridable -- add
--adaptive-polish/--no-adaptive-polish (matching --restore-faces; works without
--auto too) so the polish can be disabled or used manually. _apply_auto overrides
exactly the three content-adaptive modes (pipeline, restore-faces, adaptive-
polish); --unsharp/--humanize stay independent fixed filters.
cv2-only, no new deps. Threaded through invisible/all (not batch).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three P2 cleanups from a library-wide review.
Detection -- single C2PA_AI_VENDORS registry (noai/constants.py):
- C2PA_ISSUERS, SYNTHID_C2PA_ISSUERS, and identify._ISSUER_PLATFORM now derive
from one C2paAiVendor table, so adding a C2PA vendor is one entry instead of
edits in three places across two files. Behavior-identical (262 detection
tests pass; the kept `needle` field is load-bearing -- it differs from `org`
for Google and ByteDance, with no mechanical derivation).
Code-health:
- region_eraser.erase_lama now accepts grayscale/BGRA like erase_cv2 (it
crashed on grayscale and silently dropped alpha on BGRA). +2 regression tests.
- batch frees the device cache between images via a shared try_empty_device_cache
helper (generalized from the MPS-only _try_clear_mps_cache, now reused by both
the MPS->CPU fallback and the batch loop).
- batch gained --controlnet-scale (parity with invisible/all).
CI / packaging:
- publish.yml uploads via `uv publish` (PyPI trusted publishing over OIDC),
replacing pypa/gh-action-pypi-publish so uploads no longer depend on that
action's bundled twine accepting the Metadata-Version. Workflow filename +
pypi environment unchanged, so PyPI's trusted-publisher entry still matches.
- hatchling pin relaxed <1.28 -> <1.31 (verified against hatch's changelog:
1.30.0 made Metadata 2.5 the default, 1.30.1 reverted to 2.4; 1.27-1.29 were
always 2.4). Kept as belt-and-suspenders so the first uv-publish release ships
2.4, isolating the uploader swap from the metadata-version bump.
Docs (CLAUDE.md, pyproject) synced; corrected the inaccurate "hatchling 1.28+
emits 2.5" note.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add `auto_config.plan(image_path) -> AutoConfig`, the first step of the
invisible/all pipeline: it inspects the input image (before the diffusion model
loads) and picks the quality modes so the run adapts to content. Quality-priority
routing -- ControlNet (text/face-structure preservation) is the default, skipped for
plain SDXL only on a clearly structure-less image; GFPGAN face restore when a face is
present; a mild sharpen + grain polish when a smoothing pass ran. Exposed as `--auto`
on `all`/`invisible` (`_apply_auto`; explicit flags override via click's parameter
source). Not wired into batch (its engine is cached per-mode).
Detection is cv2-only and torch-free (~100 MB peak RSS, a few ms): OpenCV YuNet
(`cv2.FaceDetectorYN`, MIT, 232 KB model bundled in assets/) for faces, a Canny
edge-density + MSER heuristic for text/structure (a rough Phase-1 placeholder; DBNet
via cv2.dnn is the planned upgrade). ZERO new pip deps. Designed to run wherever the
pipeline runs -- the raiw.cc Modal GPU worker -- never on the 512 MB web host.
Real-ESRGAN-via-Spandrel upscaling (a new `esrgan` extra) and an adaptive
Laplacian-variance polish are deferred to later phases.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two quality knobs for the SDXL invisible pass:
- min_resolution floor (default 1024, --min-resolution): small inputs are
upscaled to a 1024px long-side floor before diffusion, since SDXL img2img
distorts on a tiny latent (a 381x512 portrait wrecks at native). The output
is restored to the original input size, so it is a transparent quality boost;
it adds time/memory on small inputs. 0 disables. Extends the pure _target_size
helper (now cap-or-floor-or-native, min skipped on a min>max misconfig),
unit-tested without a model.
- unsharp post-filter (humanizer.unsharp_mask, --unsharp, opt-in default 0):
applied LAST, after the GFPGAN face pass (a pre-GFPGAN sharpen would be
smoothed back over), to counter the soft/over-smoothed look that diffusion +
restoration leave behind (an AI tell). Pairs with --humanize (grain).
Both threaded through invisible/all/batch + the module-level helper. Verified
end-to-end on a 381x512 portrait: upscaled to 1024, sharpened, restored to
381x512.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Both content-preservation features are now flagged EXPERIMENTAL and opt-in.
--pipeline controlnet was already opt-in (default=default); --restore-faces
flips from on-by-default to OFF by default, matching the repo's prior pattern
for experimental preservation passes (the removed protect_text/protect_faces).
- cli.py: --restore-faces/--no-restore-faces default False; EXPERIMENTAL in the
--restore-faces / --controlnet-scale / --pipeline help; batch default False.
- invisible_engine.py: remove_watermark restore_faces default False + docstring.
- CLAUDE.md / README.md / docs/synthid.md: label both experimental/opt-in.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an optional, commercial-safe face-restoration post-pass that recovers
face identity the diffusion removal pass drifts (canny holds structure, not
likeness) while still scrubbing the pixel watermark in the face regions.
- face_restore.py: GFPGANer singleton (CPU unless CUDA), the basicsr
torchvision.transforms.functional_tensor shim, and the pure feather
_composite_faces helper (unit-tested without the model). GFPGAN
re-synthesizes each face from a StyleGAN2 prior, so composited face pixels
are GAN-generated (no watermark, no pixel-copy) -- oracle-clean at weight 0.5
with identity preserved.
- InvisibleEngine.remove_watermark: restore_faces / restore_faces_weight,
best-effort, auto-skips when the extra is absent or no face is detected.
- CLI --restore-faces/--no-restore-faces + --restore-faces-weight on
invisible/all/batch (on by default).
- restore extra (gfpgan/facexlib/basicsr), numpy<2-pinned (scipy<1.18,
numba<0.60) and kept out of `all`; basicsr needs Python <3.13 + setuptools<69
to build, so pin .python-version 3.12.
Commercial-safe: GFPGAN Apache-2.0, RetinaFace MIT. The CodeFormer alternative
is non-commercial and is not shipped. The earlier IP-Adapter FaceID layer was
removed (footgun: needs high strength, corrupts faces at the low removal
strength).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add `--pipeline controlnet` (SDXL base + xinsir canny ControlNet via
StableDiffusionXLControlNetImg2ImgPipeline): the canny edge map conditions the
img2img regeneration so text and face STRUCTURE stay sharp, while the watermark
is still removed by the regeneration (`strength`) -- no original pixels are
copied or frozen, so SynthID does not survive. Oracle-verified clean on OpenAI
with better text/structure fidelity than plain img2img at equal strength.
`--controlnet-scale` tunes structure preservation; fp32 on mps/cpu (fp16-fixed
VAE on cuda/xpu). Shares the img2img runner (live progress + MPS->CPU fallback)
and the fp16-VAE-fix / device-move helpers with the default pipeline.
Remove the superseded subsystems -- ctrlregen (SD1.5 clean-noise),
text-protection (differential / region-hires) and face-protection: they either
destroyed real content or shielded the watermark by re-using original pixels.
controlnet replaces them by regenerating everything under edge conditioning.
Canny preserves face structure but not identity; face IDENTITY is a separate
face-restoration post-pass (CodeFormer/GFPGAN), researched + prototyped but not
yet shipped. An IP-Adapter FaceID attempt was built and removed (footgun: needs
high strength, corrupts faces at removal strength).
Docs: docs/controlnet-removal-pipeline-research.md, scripts/controlnet_sweep.py.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The default img2img strength is now chosen from the detected SynthID vendor
(C2PA issuer) instead of a single fixed 0.30: OpenAI gpt-image -> 0.10, Google
Gemini -> 0.15, unknown source -> 0.15. Explicit --strength always wins.
Basis: an oracle-verified June 2026 controlled study (clean v0.8.6, text/face
protection OFF, per-image openai.com/verify or Gemini-app verdict). OpenAI's
SynthID clears at 0.05 across 1024-1600 px (n=4, resolution-independent);
Google's is ~3x more robust and needs 0.15 on the capped-1536 path (n=4). The
dominant factor is the VENDOR, not resolution. The earlier single 0.30 default
and the "resolution dependence" lore came from contaminated tests run with the
protect-text bug ON (issue #14) -- re-running those same 1600x1600 images clean
removes SynthID at 0.05.
`vendor_for_strength(path)` reads metadata.synthid_source on the ORIGINAL input
and is threaded through cli (invisible/all/batch) -> invisible_engine ->
watermark_remover -> resolve_strength(strength, profile, vendor), so display and
execution use the same vendor (the engine sees a temp path whose C2PA the visible
pass already stripped, so detection must happen in the CLI on the pristine
source). Caveat: Google's 0.15 was validated only on --max-resolution 1536;
native 2816 Gemini was not locally measurable (OOM on Apple Silicon) and is
pending GPU validation on raiw.cc.
Docs: docs/synthid.md sections 2.2/4.4/5.2 corrected (the contaminated
resolution-dependence findings replaced with the clean oracle-verified table);
README and CLAUDE.md updated; CLI --strength help reflects the adaptive default.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Both text and face protection were shielding SynthID from removal. The
text-protection high-res re-scrub regenerates pixels at an upscaled resolution
where the per-region pass may not be strong enough to re-destroy the SynthID
payload, allowing it to survive in text areas. Face protection has an even more
direct mechanism: it pastes back the original (pre-diffusion, watermarked) face
pixels after the global pass, guaranteeing SynthID survives in face regions
regardless of strength.
Both --protect-text and --protect-faces are now off by default and opt-in.
Rename from --no-protect-text / --no-protect-faces to --protect-text /
--protect-faces. Extract shared click.option decorators to module-level
constants (_protect_text_option, _protect_faces_option) to eliminate
copy-paste between cmd_invisible and cmd_all.
Add docs/synthid.md: primary-source-cited technical reference for SynthID-Image
covering mechanism (post-hoc encoder/decoder, 136-bit payload, pixel-space, no
model-weight modification), robustness numbers (arXiv:2510.09263: ~99.98% TPR
at 0.1% FPR across 30 transforms), removal attacks and forensic detectability
(arXiv:2605.09203: all 6 attacks detectable >98% TPR@1%FPR), detectability
limits, oracle scope, adoption landscape, and practical implications including
the protect-text/faces SynthID-preservation finding.
Verified June 2026 on gpt-image 1600x1600 via openai.com/verify: with
--protect-text SynthID detected; without, SynthID removed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An oracle-verified GPU strength study (Modal A100, native res, Gemini-app
'Verify with SynthID', n=3 fresh Gemini images, protect_text/faces off) found the
current Google SynthID survives strength 0.10/0.15/0.2 and is removed only at 0.3.
The previous 0.10 default (set from an n=1 result) no longer clears it -- Google
hardened SynthID and the threshold has climbed 0.05 -> 0.10 -> ~0.3. Bump
DEFAULT_STRENGTH to 0.30; OpenAI/ChatGPT carry C2PA not SynthID, so 0.10 is plenty
there (pass --strength 0.10). Note protect_text shields the text regions SynthID
hides in (use --no-protect-text for full removal on text-heavy images).
The same study found ctrlregen at clean-noise strength DESTROYS real images
(hallucinated micro-text in smooth regions), with no usable middle setting, so the
literature's 'clean-noise is the lever' did not hold empirically. Flag ctrlregen
EXPERIMENTAL in the CLI --pipeline help, README, and watermark_profiles; SDXL
img2img at ~0.3 stays the shippable path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mirrors --no-protect-text: when the image has no people, skip loading and
running the YOLO face detector entirely. The heavy extract+blend already only
ran when a face was found, but the detector itself always loaded+inferred to
decide; this flag lets callers skip that fixed cost.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cli.py now emits plain ASCII through a small click.echo shim
(_Console / _Table / _Progress) instead of rich: no colors, markup tags,
panels, progress bar, or Unicode glyphs (Warning: / -> / ... and dropped
checkmark/cross marks). identify and metadata tables render as indented
plain lines.
- drop rich from dependencies (pyproject.toml + uv.lock)
- __init__: set TRANSFORMERS_VERBOSITY=error (setdefault) plus a warnings
filter so the transformers Siglip2ImageProcessorFast deprecation no
longer prints at CLI startup (it fires from the eager noai import)
- TestGpuHintMarkup: the [gpu] hint is now printed verbatim; docstring updated
- CLAUDE.md: replace the obsolete rich-markup lesson, note the verbosity fix
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Visible-watermark work across all three corner-mark engines plus a committed,
reproducible alpha-build pipeline (scripts/visible_alpha_solve.py) fed by committed
solid black/gray/white captures.
- jimeng: new "即梦AI" wordmark remover (reverse-alpha + thin residual inpaint,
always NCC-aligned -- the mark re-rasterizes/jitters per image). Detect via glyph
silhouette NCC (0.45 threshold; does not cross-fire with Doubao). Registered in the
visible-mark catalog; `visible --mark jimeng` / `--mark auto`.
- doubao: fix a real production defect -- the shipped remover left a READABLE
"豆包AI生成" outline on real samples while detect() returned conf 0.0 (fooled by a
thin outline), so the test passed and the "56/56 clean" claim was detector-measured,
not visual. Root cause: under-estimated alpha + fixed-geometry-no-inpaint + tight
locate box. Rebuilt alpha (careful gray-self solve), always-align, thin inpaint,
widened locate box -> readable outline becomes faint texture-level traces.
- gemini: rebuild gemini_bg_{96,48} from our own controlled captures (validated NCC
0.9998 vs the prior third-party asset); removal re-verified clean, no behaviour change.
- tests: add textured-shift regression to both engines (guards the align-on-shift path
the Doubao defect exposed; lesson: a detector-only removal test is insufficient,
assert visual residual).
- docs: CLAUDE.md, README, capture READMEs and docstrings synced; stale
"exact/pixel-exact/56-clean" claims removed.
Also includes a SynthID label-wording clarification in identify.py/cli.py
("SynthID pixel watermark" -> "SynthID watermark, inferred from C2PA metadata").
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Apply fixes from a full-repo review (code, tests, docs).
Security / correctness:
- Clamp attacker-controlled PNG/caBX chunk lengths to the remaining file
size in metadata.py and noai/c2pa.py (a malformed length no longer drives
a multi-GB read); skipped chunks seek instead of read.
- noai/isobmff.strip_c2pa_boxes is now fail-safe on a malformed box: return
the original bytes with a warning instead of silently truncating the tail,
so metadata --remove can no longer emit a corrupt file.
- doubao_engine._fixed_alpha_map clamps the glyph box to the image (no crash
on degenerate width-vs-height).
- watermark_remover._run_region_hires gates the phaseCorrelate offset on
response and magnitude (a spurious shift no longer garbles text) and drops
the generator after a CPU fallback (no MPS/CPU device mismatch).
Robustness:
- gemini_engine, doubao_engine, region_eraser normalize grayscale and RGBA
inputs to BGR at the engine entry points.
- image_io.imwrite returns False on an unwritable path (matches cv2).
- invisible_engine guards a None imread result before use.
- trustmark_detector._decoder uses a double-checked threading lock.
- ctrlregen.tiling.tile_positions raises on overlap >= tile.
- humanizer chromatic shift no longer wraps opposite-edge pixels.
- identify OpenAI caveat keyed on the normalized vendor, not a substring.
- Remove the dead "visible --detect-threshold" CLI option.
- publish.yml verifies the release tag matches the package version.
Docs:
- README strength 0.05 to 0.10; .env.example HF_TOKEN marked optional;
doubao_capture README updated to reverse-alpha-only; CLAUDE.md synced with
the new behaviors and the batch command.
Tests: new test_security_clamp.py for the read clamp and isobmff fail-safe;
erase CLI coverage; integrity-clash rule 2 end-to-end; multi-tag EXIF
survival and cross-format strip guards; channel/size, tiling, humanizer, and
imwrite regressions. Full suite 493 passed, 2 skipped; ruff and pyright src/
clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The `metadata` command handles more than images: `remove_ai_metadata` strips
C2PA / AIGC provenance from MP4/MOV/M4V/M4A and from WebM/MP3/WAV/FLAC/OGG via
ffmpeg. But the help said "from images" and the shared `_validate_image` call
printed "Warning: .mp4 may not be supported" on exactly those supported
containers. The argument's `exists=True` already enforces the file exists, so
the validation call only added the wrong warning here.
Update the docstring to list the real format coverage and drop the
image-only validation from this command. The image commands keep it.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Raise default SynthID-removal strength 0.05 -> 0.10 (current Google SynthID)
The old default (0.04/0.05) no longer removes the CURRENT Google SynthID (Nano
Banana / Gemini 3): verified 2026-05-30 via the Gemini 'Verify with SynthID'
oracle on a real image -- 0.05 still detected, 0.10 not detected (OpenAI's was
already cleared at 0.05). Add DEFAULT_STRENGTH=0.10 in watermark_profiles, route
the engine + CLI defaults to it. At 0.10 small text deforms more, which is why
text protection (_run_region_hires) runs by default. CLAUDE.md SynthID note
corrected. CAVEAT: n=1 Google + n=1 OpenAI; broad corpus oracle validation
pending (task tracked).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Drop unused LOW/MEDIUM/HIGH strength profiles; CLI --strength defaults to DEFAULT_STRENGTH
The fixed strength presets (and get_recommended_strength) were dead -- nothing in
the pipeline used them, only tests. One knob now: DEFAULT_STRENGTH (0.10),
overridable per-call via the CLI --strength flag, which now defaults to that
constant (single source of truth). Removed the WatermarkRemover.LOW/MEDIUM/HIGH
class attrs and the get_recommended_strength tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On RGBA inputs the CLI forced the watermark bbox alpha to 0 on save, so the
removed-sparkle area became a transparent hole that renders as a solid white
box on any non-transparent viewer. The Gemini app exports opaque RGBA, so
every user hit it. Reverse-alpha already recovers the real pixels there (and
`erase` inpaints them), so there is no artifact to hide -- the hole was the
bug, introduced as an over-correction in d091b9f.
`_write_bgr_with_alpha` now rejoins the input alpha plane unchanged (drops the
`clear_region`/`pad` params); the `visible` / `erase` / `all` / `batch` call
sites drop the cleared-region argument and the orphaned region bookkeeping.
The registry `remove()` still returns the mark bbox (used for inpaint_residual
positioning); the CLI just no longer clears alpha with it.
Inverts the test that locked in the old behavior into a #30 regression guard
(watermark-region alpha stays opaque, no pixel forced transparent). Verified
end-to-end on a real Gemini RGBA export: sparkle gone, zero transparent
pixels, clean over a white background.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(trustmark): gate detection on re-encode durability to kill false positives
TrustMark's wm_present flag is a BCH validity check that spuriously
validates on a content-correlated fraction of un-watermarked images
(AI textures trip it more than camera photos). On a 1343-image set all
20 raw detections were false, several on Gemini/OpenAI/Doubao output that
cannot carry Adobe's watermark, with random-bytes secrets.
A genuine TrustMark is a durable soft binding that survives re-encoding,
so detect_trustmark now re-decodes after a mild JPEG round-trip and
requires the same schema both times. Every observed false positive
collapsed under this gate; the second decode runs only on the rare hit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(identify): Samsung Galaxy AI, FLUX, ByteDance C2PA; fix C2PA substring FP
Detection extensions verified on real signed files (2026-05-29):
- Samsung Galaxy AI: signer attribution via a new _SIGNER_C2PA_PLATFORM
(Samsung Galaxy / ASUS Gallery) kept separate from the capture-camera
_DEVICE_C2PA_PLATFORM so a Galaxy AI edit (device cert + AI source type)
does not trip the camera-vs-AI integrity clash. Plus metadata.samsung_genai:
the proprietary genAIType marker in PhotoEditor_Re_Edit_Data, a medium-
confidence AI-editing signal (samsung_only branch).
- Black Forest Labs (FLUX) and ByteDance Volcano Engine (Doubao/Jimeng)
added as C2PA issuers + issuer->platform mappings.
- fix: C2PA presence required only the bare 4-byte 'c2pa' substring, which
false-positives on compressed pixel data (a recompressed PNG IDAT re-flagged
C2PA after its manifest was correctly stripped). New c2pa_marker_in() requires
the JUMBF wrapper (jumb+c2pa) or the C2PA uuid box; applied in identify +
metadata. Verified: all 535 real C2PA files carry jumb.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(doubao): gate detection on text structure to cut ~95% of false positives (#23)
Coverage alone over-fired: any textured bottom-right corner cleared the
threshold, so the detector false-positived on ~28% of arbitrary images.
The real '豆包AI生成' mark is six glyphs in one row, so detect now also
requires the text-structure signature (_glyph_structure): many connected
components, no single dominant blob, concentration in a thin horizontal
band. False positives dropped 343 -> 17 across the corpus while keeping
real-mark recall and the doubao-1.png sample. Also accept a no-op force
kwarg for remover-interface symmetry.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(samsung): add Samsung Galaxy AI visible-badge remover
New samsung_engine.py removes the bottom-left sparkle + localized
'AI-generated content' badge that Galaxy AI tools stamp. Mirrors the
Doubao locate->mask->inpaint pattern but bottom-left, with a dual-polarity
top-hat mask (the badge is light-on-dark or dark-on-light). Detection gates
on a band + left-anchor signature (the Doubao CJK-component gate does not
transfer: Latin badge letters connect into few blobs). Explicit-only --
tuned on few real badges with a ~4% FP floor, so it is not used in auto.
Synthetic byte-blob fixtures (real badges are user content, not shipped).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(visible): unified known-watermark registry + LaMa inpaint backend
watermark_registry.py is a single catalog of known visible marks, each
tying {usual location, in_auto flag, recovery strategy, detect adapter,
remove adapter}: gemini (reverse-alpha, exact), doubao, samsung. cmd_visible
is now registry-driven (best_auto_mark for --mark auto; mark_keys() feeds the
CLI choices) -- the per-mark _run_doubao/_run_samsung helper branches are gone.
Cross-engine confidences are not comparable, so the gemini adapter applies the
corpus-validated 0.5 sparkle threshold for auto arbitration (its engine flag is
loose and weakly fired ~0.36 on Doubao text, hijacking auto).
--backend auto|cv2|lama chooses background reconstruction for the mask-based
marks; auto = LaMa when onnxruntime is present, else cv2. For LaMa the mask is
the FILLED glyph bounding box (sparse glyph masks leave anti-aliased edges
behind). cv2 stays the zero-dependency fallback.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: watermark registry, Samsung/FLUX/ByteDance detection, LaMa backend, trustmark gate
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(doubao): exact reverse-alpha removal from captured alpha map
The Doubao '豆包AI生成' mark is a fixed semi-transparent white overlay, so
given its alpha map the original pixels are recovered exactly:
original = (wm - a*logo)/(1-a) -- no inpaint hallucination.
The alpha map + logo colour were solved from real black+gray Doubao captures
on a controlled background: on black captured = a*logo, and the black/gray pair
solves a per-pixel without assuming the logo colour (a_max~0.65, logo near-white);
the white capture cross-validates (mark vanishes to a flat fill). Bundled as
assets/doubao_alpha.png + geometry constants.
remove_watermark_reverse_alpha applies it scaled to image width; exact at the
captured width, so the registry routes doubao through it only when
reverse_alpha_available (width within the calibrated band) and the mark is
detected, falling back to mask inpaint (cv2/LaMa) otherwise. A light residual
inpaint cleans the sub-pixel rescaling error. Add captures at more resolutions
to widen exact coverage.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(visible): reverse-alpha only -- drop inpaint removal + heuristic detection
Per the principle that we only remove/detect what we can do exactly, the
visible-mark path is now reverse-alpha only:
- Doubao detect is reverse-alpha-consistent: match the bundled alpha glyph
silhouette against the corner via TM_CCOEFF_NORMED (DETECT_NCC_THRESHOLD 0.4)
-- keys on the '豆包AI生成' SHAPE, not coverage/structure heuristics. FP
7/1243 (0.6%). Removes the cv2 inpaint path + the _glyph_structure gate.
- Registry is reverse-alpha only: dropped the cv2/LaMa backend (_glyph_remove,
_lama_box_inpaint, default_backend, --backend) and the Samsung entry. Doubao
outside the alpha resolution band is skipped, never inpainted.
- Removed samsung_engine.py + tests + --mark samsung (no alpha map captured;
Samsung C2PA/genAIType metadata detection in identify is unaffected).
- The universal erase --region (cv2/LaMa) is unchanged -- arbitrary-region
inpainting stays a user-directed tool, separate from the known-mark registry.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(doubao): NCC sub-pixel alignment -> reverse-alpha at any resolution
A pure width-scale of the captured alpha map is only sub-pixel-accurate at the
captured width and leaves a faint ghost elsewhere. remove_watermark_reverse_alpha
now registers the alpha glyph to the actual mark via a TM_CCOEFF_NORMED
scale+position search (_aligned_alpha_map) before inverting the blend, so the
single 2048 capture works at any resolution -- verified clean on the 1773x2364
(3:4) corpus size, the biggest coverage gap (23 files).
reverse_alpha_available is now just 'asset present' (no width band); the registry
still gates removal on detect so a clean corner is never touched. Drops the
_ALPHA_WIDTH_TOLERANCE gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(doubao): keep native recovery exact -- fixed geometry at captured width
Integer-pixel NCC alignment landed ~1px off at the captured width, degrading the
otherwise-exact native reverse-alpha (synthetic recovery error 0.94 -> 1.39).
remove_watermark_reverse_alpha now uses exact width-relative geometry within
_ALPHA_NATIVE_BAND of the captured width and the NCC search only off it -- best
of both: native back to 0.94, other resolutions still aligned.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(doubao): harden alignment -- try fixed+aligned, keep least residual (56/56)
On a faint/busy-background mark the NCC alignment peak can wander a few px off
the true mark and leave a residual (2/56 real corpus files). Off the captured
width, remove_watermark_reverse_alpha now builds BOTH the fixed-geometry and the
NCC-aligned alpha map, applies each, and keeps whichever leaves the least
residual mark (re-detect confidence on the bare reverse-alpha) -- geometry wins
on faint marks, alignment on clear ones, no magic threshold. Real-file round-trip
now removes 56/56 detected Doubao clean across every corpus resolution (was 54).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(doubao): skip residual inpaint at native width for exact recovery
At the captured width the fixed-geometry reverse-alpha is pixel-exact, so
inpainting over it only replaced exactly-recovered interior pixels with a
cv2 hallucination -- measured worse on a textured background (native error
vs true bg 1.6 reverse-alpha-only vs 2.6 with the old always-on
full-footprint inpaint). Native now returns the bare recovery untouched;
off-native, where NCC alignment is only sub-pixel-approximate, the footprint
inpaint stays to clean the seam. Real round-trip still 56/56 across all
corpus resolutions; negatives 0/60, Gemini unaffected.
Add test_native_returns_exact_reverse_alpha_no_inpaint as the regression
guard. Sync CLAUDE.md + README (the table cell and prose described the
pre-NCC "skipped off native / cv2-LaMa" behavior, now stale). Gitignore the
session scheduled_tasks.lock, and add the text-protection research note.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(device): support xpu backend
* Fall back to CPU seed generator when device RNG unsupported (xpu)
Some torch-xpu builds have no device-side RNG, so torch.Generator(device="xpu")
raises when --seed is used. _make_seed_generator tries the device generator and
falls back to a backend-agnostic CPU generator. Adds a fallback unit test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Victor Kuznetsov <kuznetsov.va@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A bare "unknown" verdict reads as the tool being broken. Print a one-line note
right under the verdict explaining that no locally-readable AI signal was found,
that this is not the same as clean (metadata is often stripped), and that
SynthID-class pixel watermarks have no local detector. The why was previously
only in the caveats section below.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Make `pyright src/` strict-clean via a hybrid: pure-logic files are fully typed
(piexif gets a local typings/ stub; PIL info-dict loops guard isinstance(key, str);
progress returns Callable[..., None]; availability checks use importlib.util.find_spec
instead of unused imports), while the irreducibly-untyped cv2/torch/diffusers boundary
files carry a documented per-file `# pyright:` relax pragma (or a ctrlregen
executionEnvironment) that disables only the unknown-type rules. Public ndarray-returning
signatures on the relaxed engines are annotated NDArray[Any] so strict consumers (cli.py)
stay clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mirror protect_faces: protect_text defaults to True in invisible_engine and
watermark_remover, so the SDXL pipeline detects text per image and switches to
Differential Diffusion only when glyphs are found. Text-free inputs fall back to
plain img2img with no differential-pipeline load, so the autonomy is free. The
CLI now exposes a single off-switch --no-protect-text instead of the positive
flag, keeping the interface minimal.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
SDXL img2img regenerates every pixel, so small text and CJK glyphs deform
at the strengths that defeat SynthID (issue #21). With --protect-text a
CJK-native PP-OCRv3 detector (2.4 MB ONNX, cv2.dnn, no torch, cached on
first use) locates text regions and the pass switches to the SDXL
Differential-Diffusion community pipeline: a per-pixel change map keeps
text regions largely intact while the background is regenerated to strip
the watermark. Gated to the SDXL default model; falls back to plain
img2img with a warning when unavailable.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Surface contradictions between independent provenance signals instead of
collapsing to a single verdict -- a strong tell of spoofed, transplanted, or
laundered metadata. Inspired by arXiv:2603.02378.
Two rules in the new _integrity_clashes helper:
- Conflicting AI-origin attributions: two or more distinct AI vendors named by
independent generator stamps (e.g. a C2PA OpenAI manifest on an image whose
EXIF says Make="Ideogram AI").
- Camera + AI: a camera-capture C2PA device (Pixel/Leica/Sony/Nikon/Truepic)
coexisting with an AI-generation marker -- a genuine capture is not AI.
High-precision by design: only hard generator stamps feed it (C2PA issuer when
the source is AI, SynthID proxy, EXIF/XMP generator, IPTC AISystemUsed, xAI,
AIGC). The fuzzy visible sparkle and the open invisible watermark are excluded
-- the latter can be a by-product of our own SDXL removal pass. Vendor
normalization (_vendor_of over _AI_VENDOR_TOKENS) keeps consistent signals from
clashing (C2PA "Google (Gemini)" + SynthID-Google agree); the C2PA vendor is
read from the issuer attribution, not the resolved platform, so a camera label
like "Google Pixel" cannot mis-normalize to an AI vendor.
Surfaced as ProvenanceReport.integrity_clashes (red in the table view, included
in --json). 19 new tests; all real single-origin fixtures (chatgpt/firefly/
doubao/grok/mj) verified to produce zero clashes (false-positive guard).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two CLI/IO robustness bugs surfaced by issues #17 and #19.
#17 -- non-ASCII image paths (Chinese/Cyrillic/accented) failed on Windows:
cv2.imread/imwrite use the platform ANSI code-page API, so the decode came back
empty with a "can't open/read file" warning. New image_io.imread/imwrite route
through np.fromfile+cv2.imdecode / cv2.imencode+tofile (Unicode-safe, byte-
identical output, cv2.imread None-semantics preserved); all 8 cv2 read/write
call sites now go through it. Behavior-neutral on macOS/Linux (already accept
UTF-8 paths), so the fix is correct-by-construction for the Windows-only bug.
#19 (incidental) -- rich parsed the "[gpu]" in the GPU-extra install hint as a
style tag and dropped it, so the printed command was the un-installable
"pip install 'remove-ai-watermarks'". Escaped as \[gpu] at both call sites.
Tests: test_image_io.py (non-ASCII round-trip, alpha, missing/empty/garbage
semantics); test_cli.py::TestGpuHintMarkup (install hint keeps the extra).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add deterministic, CPU-only removal of the visible Doubao "豆包AI生成" mark and
a position-agnostic region eraser for any other visible watermark/logo.
- doubao_engine.py: locate (geometry, scales with width) + polarity-aware
white-top-hat glyph mask + cv2 inpaint; coverage-gated detection and a
dense-text safety guard. No GPU, ~30ms.
- region_eraser.py + `erase` command: inpaint arbitrary --region box(es).
Default cv2 backend (no deps); optional big-LaMa via onnxruntime (`lama`
extra, Carve/LaMa-ONNX, model downloaded on first use, never bundled).
- cli `visible --mark auto|gemini|doubao`: auto routes by detector confidence.
- tests for both engines; seed previously-unseeded CLI image fixtures to stop
the Doubao detector flaking on random corners.
- .gitignore: doubao_capture/{seeds,captures} scratch (alpha-map calibration).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The invisible pipeline force-downscaled inputs >1024px to 1024 before
diffusion, then upscaled the result back -- a lossy round-trip that was
the main cause of the quality loss reported in #10. The hosted raiw.cc
backend (fal fast-sdxl) does no pre-downscale, and at strength ~0.05
SDXL img2img doesn't need it.
Default is now native resolution (max_resolution=0). New --max-resolution
flag (invisible / all / batch) re-introduces an opt-in long-side cap only
to bound GPU/MPS memory on very large inputs.
Addresses #10. End-to-end quality/removal not re-verified locally (no GPU
here); matches raiw-app's proven production config.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`cv2.imread(..., IMREAD_COLOR)` was silently stripping the alpha channel
on RGBA inputs, and `cv2.imwrite` then wrote opaque 3-channel PNGs — so
images with transparent backgrounds came back with an opaque-black (or
white) background and the sparkle area baked in as a solid blob.
Read the source with `IMREAD_UNCHANGED`, keep the alpha plane out of the
detection/inpaint path (those still operate on BGR), and rejoin alpha at
save time. The detected watermark bbox is also zeroed in the alpha plane
so the sparkle region becomes transparent rather than an opaque artifact.
Applies to `visible`, `all`, and `batch` modes. RGB-only inputs and JPEG
outputs are unaffected.
Research found one locally-fillable detection gap: Stable Diffusion, SDXL,
and FLUX all embed an open DWT-DCT watermark via the invisible-watermark
(imwatermark) library -- a PUBLIC decoder, no secret key, unlike SynthID.
New invisible_watermark.py decodes the known fixed patterns (verified
against upstream source: diffusers SDXL WATERMARK_MESSAGE, FLUX.2
src/flux2/watermark.py, and the 'StableDiffusionV1' default string) and
identify() reports the scheme as a high-confidence signal.
Verified locally end-to-end: embedding SDXL's exact 48-bit message and
decoding it back recovers 48/48 bits; a clean image and our own fal-SDXL
outputs decode to ~21/48 (no match). Caveat baked into the report: the
watermark is fragile -- gone after JPEG q90 -- so it confirms origin only
on pristine files; absence is never proof.
imwatermark is an optional dep (extra 'detect'; pulls non-headless opencv),
so the import is guarded and the signal is skipped when absent. CLI
--no-visible now means metadata-only (skips both pixel-domain detectors).
Also records the broader watermarking landscape in CLAUDE.md: which
services are locally detectable (SD/SDXL/FLUX), C2PA-covered (Bing/Canva/
Getty/Shutterstock unsampled), or proprietary-only like SynthID (Amazon
Titan/Nova, Kakao). Midjourney embeds neither C2PA nor an invisible mark.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New 'identify' command and identify.py module: upload an image, get one
ProvenanceReport answering where it was made and what watermarks it
carries. Aggregates every locally-readable signal:
- C2PA Content Credentials -> generating platform (issuer + generator).
- IPTC digitalSourceType 'Made with AI' (Meta and others).
- Embedded SD/ComfyUI generation parameters (local pipelines).
- SynthID metadata proxy (Google / OpenAI C2PA companion).
- Visible Gemini sparkle (cv2 fallback for the stripped-metadata case),
promoted only at confidence >= 0.5 (corpus-tuned: Gemini sparkles
score >= 0.56, non-sparkle <= 0.49).
is_ai_generated is True or None, never asserted False -- stripped
metadata leaves no local proof of a clean origin, so absence of signals
is reported as 'unknown' with an explicit caveat. The SynthID *pixel*
watermark remains locally undecodable; the report says so.
Non-PNG containers (JPEG/WebP/AVIF/HEIF/JXL) get the same issuer +
generator attribution via a binary scan (the caBX parser is PNG-only).
The cv2 dependency is isolated in gemini_engine.detect_sparkle_confidence
so identify.py stays type-clean. CLI supports --json and --no-visible.
Validated against the 109-image corpus: 14/14 positives flagged AI,
93/94 negatives clean (the one 'neg' flagged is a Meta image that
genuinely carries the IPTC tag -- correct), zero true errors.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Detect SynthID-bearing images via their C2PA companion: a manifest signed by a
SynthID-using vendor (Google/OpenAI) on AI-generated content implies an
invisible SynthID pixel watermark. Verified end-to-end against the vendor
oracles (openai.com/verify, Gemini "Verify with SynthID").
- metadata: synthid_source() + synthid_watermark verdict in get_ai_metadata,
surfaced as a `metadata --check` callout. Format-agnostic (PNG caBX parser +
JPEG/WebP/AVIF/HEIF/JXL binary scan).
- constants: SYNTHID_C2PA_ISSUERS {Google, OpenAI}; +opened/placed actions.
- c2pa: single CBOR-aware parser (_cbor_text_after) replaces glitchy regex
(fixes fGPT-4o claim_generator); removed duplicate _scan_png_c2pa_chunk from
metadata; shared synthid_verdict / synthid_vendors_in helpers.
- corpus: scripts/synthid_corpus.py ingest tool + data/synthid_corpus/
(manifest tracked, images gitignored) for a labeled reference set.
- tests: +38 across C2PA parser internals, extract/inject round-trip, ISOBMFF
container stripping, all IPTC AI markers, and invisible watermark strength
tiers (SynthID/StableSignature/TreeRing/StegaStamp/RingID/RivaGAN/...).
Pixel-level SynthID detection remains out of reach locally (Google's decoder is
proprietary); a from-scratch spectral pilot confirmed it does not separate real
content. See CLAUDE.md for the full evaluation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SD-1.5 dreamshaper at 768 px did not defeat SynthID v2 on Gemini 3 Pro
outputs (verified May 2026 via Gemini app's "Verify with SynthID"). Switch
the default invisible engine to SDXL at 1024 px, matching the raiw-app
production config (strength 0.05, steps 50). Drop the SD-1.5 pipeline.
Metadata layer: add C2PA UUID and IPTC AI marker byte-scan detection
across all formats, plus an ISOBMFF box walker (noai/isobmff.py) that
strips top-level C2PA uuid and JUMBF jumb boxes from AVIF/HEIF/JPEG-XL
containers without re-encoding.
README gets a Legal table and a Threat-model section about SynthID v2's
136-bit payload. CLAUDE.md tracks the SD-1.5 regression as historical
context.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Victor Kuznetsov