Compare commits

...
162 Commits
Author SHA1 Message Date
Victor Kuznetsov 4241a01659 Release 0.45.1 2026-10-08 16:42:57 -07:00
Victor Kuznetsov c37378b347 Merge pull request #117 from dichiara19/fix/sdxl-zimage-global-offload
Stream the sdxl-zimage global stack on cards below 12 GiB
2026-10-08 16:34:43 -07:00
Victor Kuznetsov 8cb99b73b2 Fix SDXL offload placement on the current CUDA device 2026-10-08 16:17:11 -07:00
Giuseppe G. Di Chiara c45e6db3e4 Stream the sdxl-zimage global stack on cards below 12 GiB
The SDXL global stack (fp16 UNet, Canny ControlNet, both text encoders, VAE)
is ~8.8 GiB and was always moved to CUDA with .to(device), so it could not
fit an 8 GiB card and --cpu-offload never reached it. Under the Windows
display driver, WSL2 included, the overflow spilled into shared system
memory and the four-step pass ran for over ten minutes instead of failing.

Residency now follows the same rule as the Qwen stack: an explicit request
wins, otherwise the card's total VRAM decides against
RESIDENT_SDXL_GLOBAL_MIN_VRAM_GIB. Below it the Lightning LoRA is fused on
the CPU and the stack is streamed with enable_sequential_cpu_offload, the
fastest placement that completed on every run of an 8 GiB card with 16 GiB
of system RAM. sdxl-zimage joins GLOBAL_OFFLOAD_PROFILES, so the
--cpu-offload warning now fires for chroma-zimage only.
2026-10-08 23:27:55 +02:00
Victor Kuznetsov 37fb080c74 Merge pull request #116 from wiltodelta/dependabot/uv/minor-and-patch-931ef6a6bd 2026-10-08 10:30:09 -07:00
dependabot[bot] 954298aa14 Bump the minor-and-patch group with 3 updates
Bumps the minor-and-patch group with 3 updates: [python-dotenv](https://github.com/theskumar/python-dotenv), [openai](https://github.com/openai/openai-python) and [ruff](https://github.com/astral-sh/ruff).


Updates `python-dotenv` from 1.2.3 to 1.2.4
- [Release notes](https://github.com/theskumar/python-dotenv/releases)
- [Changelog](https://github.com/theskumar/python-dotenv/blob/main/CHANGELOG.md)
- [Commits](https://github.com/theskumar/python-dotenv/compare/v1.2.3...v1.2.4)

Updates `openai` from 3.22.1 to 3.24.0
- [Release notes](https://github.com/openai/openai-python/releases)
- [Changelog](https://github.com/openai/openai-python/blob/main/CHANGELOG.md)
- [Commits](https://github.com/openai/openai-python/compare/v3.22.1...v3.24.0)

Updates `ruff` from 0.16.9 to 0.16.10
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.16.9...0.16.10)

---
updated-dependencies:
- dependency-name: python-dotenv
  dependency-version: 1.2.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: openai
  dependency-version: 3.24.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: ruff
  dependency-version: 0.16.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-10-07 08:48:45 +00:00
Victor KuznetsovandClaude Opus 5.5 10c7775990 Hide maintainer skills from public skill discovery
The skills CLI discovers every SKILL.md under .claude/skills, so
provider-oracles and signal-discovery were offered by `npx skills add`
and listed on skills.sh beside the published skill. Mark both
metadata.internal, the CLI's documented hide flag, and fail the suite
on any project skill that lacks it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 20:04:18 -07:00
Victor KuznetsovandClaude Opus 5.5 3f6447b23d Record how to run the gate from an agent session
The full gate outlasts the agent's 10-minute foreground shell cap, so it must
run detached with its exit code written to a file. A stray run is stopped by
its own PID, because a pattern kill also ends other sessions' suites.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 18:25:14 -07:00
Victor KuznetsovandClaude Opus 5.5 cc63bfd7bf Release 0.45.0
Bump the package to 0.45.0 and the agent skill to 1.0.11. The skill now names
the dola and workbuddy image mark keys, which 0.44.0 rejects, so the probe's
MIN_CLI_VERSION rises to 0.45.0 and both keys carry a "since CLI" note that
the floor test reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 17:45:18 -07:00
Victor Kuznetsov 898f8be789 Add AI disclosure layouts, tighten detection and provenance warnings 2026-10-04 11:03:37 -07:00
Victor KuznetsovandClaude Opus 5.5 72752feb33 Scan uv.lock in CI and add a security policy
Add a CI job running the maintain.sh uv-secure scan, pinned to the gate by
a parity test, and a SECURITY.md with private reporting and dependency
remediation. Refresh the security-gate block record for accelerate 1.15.0.

Closes #114

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 10:33:50 -07:00
Victor KuznetsovandClaude Opus 5.5 70adddf960 Record why Qwen-Image-2.1 is not a regeneration profile
Open weights remove the hosted-only objection to the 2.x line, but the
Diffusers pipeline has no strength or latent noise-mask input and the Qwen
Research License is non-commercial (issue #111).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 10:33:49 -07:00
Victor Kuznetsov 01a2665181 Reconcile research follow-up status 2026-10-01 22:10:00 -07:00
Victor Kuznetsov 0c3e0fa415 Release 0.44.0 2026-10-01 14:24:27 -07:00
Victor Kuznetsov b3b868153f Record Google Photos checks and continue across provider accounts 2026-10-01 13:42:21 -07:00
Victor Kuznetsov 304bd071c0 Unify TC260 producers and strengthen provenance regression coverage
Share producer identities across mark routing, attribution, and pinned signers.
Preserve nested Honor TC260 fields and report unsupported RSA signatures.
Document the Google Photos checks and public sample search.

Use an SDK-signed synthetic carrier for negative EKU checks after the original
Microsoft certificates expired. Skip boundary regex scans when no C2PA
soft-binding identifier is present.
2026-10-01 11:30:32 -07:00
Victor Kuznetsov 2e26df1f21 Restore C2PA certificate validation and refresh compatible dependencies 2026-09-30 16:58:43 -07:00
Victor KuznetsovandClaude Opus 5.5 d800ec9455 Verify TC260 SecurityData label signatures against a pinned MiniMax key
MiniMax signs its TC260 label with SM3withSM2 per TC260-PG-202511A. A
verification-only SM3/SM2 (no dependency provides the SM2 curve) checks the
appendix A.1 label message; a signature under the pinned MiniMax key is a
high-confidence aigc_signature signal, a failing one is a caveat, and every
result is the aigc_signature metadata marker. Pinned by the GB/T 32905 SM3
vectors, OpenSSL's SM2 verify vectors and real MiniMax files. Over 411 local
files the only verdict-record change is the marker on nine MiniMax videos.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 15:36:08 -07:00
Victor KuznetsovandClaude Opus 5.5 ef7b972ed1 Sync the C2PA soft-binding snapshot to d36d1c1
The upstream list widened com.museblossom.contentsdefence.1 from image to
audio, image and video. Verdicts over 411 local files are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 15:36:08 -07:00
Victor KuznetsovandClaude Opus 5.5 b96ede6f44 Record the verified MiniMax TC260 SecurityData signatures
MiniMax writes a TC260-PG-202511A SecurityData object with SM3withSM2
signatures. The label signature verifies on all four local MiniMax videos
against the guide's appendix A.1 message, and the Binding signature over the
compact Bindings entry; the SM3 content hash does not match a naive selection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:16:42 -07:00
Victor KuznetsovandClaude Opus 5.5 ca59c032a1 Release 0.43.0
Bump the package to 0.43.0 and the agent skill to 1.0.10. The skill now names
the wan image and vidu video mark keys, which 0.42.0 rejects, so the probe's
MIN_CLI_VERSION rises to 0.43.0 and both keys carry a "since CLI" note that
the floor test reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 21:39:08 -07:00
Victor Kuznetsov c5e138357e Record Z-Image-Turbo global-stage rejection 2026-09-29 20:55:59 -07:00
Victor Kuznetsov 21661bea35 Reproduce re-watermarking and verify SynthID coexistence 2026-09-29 13:50:52 -07:00
Victor Kuznetsov 5cb4f2f808 Add Perth and PixelSeal benchmark oracles 2026-09-28 11:14:28 -07:00
Victor Kuznetsov 33f748b32a Record VACE and ROSE video inpainting results 2026-09-28 10:54:19 -07:00
Victor KuznetsovandClaude Opus 5.5 742a0dccab Align the remaining klein and color-restoration wording with the results
module-internals still called FLUX.2 klein pending; the sweep still counted
three inconclusive Gemini rows where only one was an explicit tool verdict,
and the removal page stated a Google chroma leak the data only warns of.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 21:24:59 -07:00
Victor KuznetsovandClaude Opus 5.5 b5fc3ef9a6 Add face and text fidelity to the FLUX.2 klein comparison
fidelity_metrics.py at each engine's clean strengths: klein keeps Latin and
Cyrillic sign lettering better on two Google carriers, but loses every face
comparison and the verified CJK text, which the 512-pixel LPIPS had hidden.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 18:59:06 -07:00
Victor KuznetsovandClaude Opus 5.5 6eb1a94799 Close the FLUX.2 klein Google measurement
The fourth Google carrier clears at 0.25, so the klein operating point with
the usual cross-source margin is 0.40, where the limiting carrier is at
LPIPS 0.270 against qwen-zimage's 0.183. klein is not a replacement global
stage for either vendor. Also record that Gemini's image-check allowance is
per account and the new "checks on similar images" quota wording.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 16:02:33 -07:00
Victor KuznetsovandClaude Opus 5.5 32791ee832 Record the FLUX.2 klein Google brackets from four Gemini accounts
Three of four Google carriers clear with klein at or below 0.20, where it
matches qwen-zimage's PSNR at 0.35, beats its LPIPS, and keeps the objects
qwen-zimage redraws. The fourth is detected at 0.10 and inconclusive at 0.20,
so there is no Google operating point yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 15:49:57 -07:00
Victor KuznetsovandClaude Opus 5.5 a103b4ec15 Record the FLUX.2 klein global-stage measurement
Stock Flux2KleinInpaintPipeline feeds the clean source back as reference
tokens, so it copies the source and OpenAI SynthID survives strength 0.30.
Without those tokens klein clears OpenAI between 0.05 and 0.20, at lower
fidelity than qwen-zimage at its floor. Google is unmeasured: the Gemini
Verify AI tool answered with a tool error or quota limit, which also casts
doubt on two of the earlier color-restoration "inconclusive" rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 15:17:26 -07:00
Victor KuznetsovandClaude Opus 5.5 7341e1a648 Record the CIELAB color-restoration measurement after regeneration
Transplanting the source's a,b channels onto qwen-zimage outputs at the vendor
floors cut CIEDE2000 on all seven tracked originals, kept every OpenAI API
verdict not_detected, but left three of four Gemini checks inconclusive and
painted old object colors onto redrawn ones. Not shipped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 14:49:22 -07:00
Victor Kuznetsov dfbaa018c6 Merge pull request #112 from wiltodelta/docs/library-research-landscape
Research sweep, provider captures, and provenance attribution fixes
2026-09-27 11:39:15 -07:00
Victor KuznetsovandClaude Opus 5.5 e310a22bd8 Compare capture paths in POSIX form on every platform
The manifest stores forward-slash paths, and str() of a Windows path uses
backslashes, so the index test failed on both Windows CI jobs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 10:11:54 -07:00
Victor KuznetsovandClaude Opus 5.5 f855c4700f Merge origin/main into docs/library-research-landscape
Main's claim_generator_platform and c2pa_vendor_platform now use the
branch's whole-word registry match, which replaces
claim_generator_word_platform. The branch's provenance veto covers every
video mark and made main's Veo-over-Sora override unreachable, so the
override is removed. Main's credential rule carries into the veto: an
invalid credential vetoes nothing. Provenance that confirms the mark's
own vendor also vetoes nothing. Over 78 local videos with a C2PA AI
claim, the only verdict changes against main are Kling/Hailuo false
positives on Veo and FLUX clips that the veto now removes. C2PA verdicts
over 1,379 files are unchanged against the pre-merge branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-26 23:52:29 -07:00
Victor KuznetsovandClaude Opus 5.5 5a0f771ed7 Close the remaining raw-byte identity scans and unify two rules
- A decoded manifest no longer takes its generator from a raw
  claim_generator CBOR value or a soft binding from raw bytes. Device tokens
  (NIKON, Pixel Camera, sony.sig) stay a byte scan, since the decoded strings
  do not carry them, but read the manifest store: an EXIF Make beside an
  unrelated AI manifest is not a verified capture. The source-kind scan keeps
  the whole buffer because it also reads IPTC digitalSourceType in XMP.
- The byte fallback's c2pa.ai-disclosure kind now uses the structured rule:
  human_validated beside prompt_guided is a generation, not an enhancement.
- Wan and Yuanbao share BottomRightAnchoredEngine instead of two copies of
  the corner gate. The Yuanbao anchor test shifted the mark outside the locate
  window, so it passed with or without the gate; it now shifts it inside.

Verdicts over 1,379 C2PA files, 468 fixture and capture files, and Wan and
Yuanbao detections over 285 images are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-26 22:19:30 -07:00
Victor KuznetsovandClaude Opus 5.5 4931b93097 Add a Higgsfield DoP capture
Higgsfield's own DoP model ("Higgsfield Lite" in the web app, job model
image2video/lite) is not exposed by the MCP connector, so the capture was made
in the web app on a JPEG frame after a first job on a PNG failed and was
refunded. Like Genjutsu, the output carries no provenance: no C2PA, no TC260
label and no visible mark, only a Lavf re-mux.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-26 21:17:27 -07:00
Victor KuznetsovandClaude Opus 5.5 509ce633a3 Judge decoded C2PA manifests on their decoded strings alone
Extend the whole-word rule to every identity lookup. Once the reader has
decoded a manifest, no identity is re-derived from its raw bytes: not the
signer, not a signer-platform token, not the AI-tool fallback for the
generator, and not the SynthID vendor inference, in identify and in
metadata.synthid_source alike. The byte scans read the manifest store only
when the reader could not open it.

Claim-generator tokens match as whole words through one shared
claim_generator_word_platform (suno no longer matches Sunoco), for images
and video alike.

The verdict now reads SynthID from the decoded C2PA info when the metadata
dict lacks it; three synthetic Google Photos tests had relied on the byte
fallback for that.

Verdicts over 1,379 C2PA files and 468 fixture and capture files are
unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-26 20:19:13 -07:00
Victor KuznetsovandClaude Opus 5.5 b02eb12cb2 Match registry tokens as whole words in decoded manifest strings
Registry tokens were raw substrings, so the Canva token matched the "Nova
Canvas" agent of an Amazon Bedrock image. Raw manifest bytes cannot take a
word-boundary rule: across the 1,379 C2PA files in the local corpus, 374 of
37,325 store matches had a letter or digit beside them, nearly all CBOR text
headers and DER tags. Decoded strings have no such framing.

- Match signer, claim-generator, agent and video-marker strings with
  registry_word_matches.
- When the reader decoded the manifest, never re-derive the signer from raw
  bytes: an unregistered signer stays unknown. Bytes decide only for
  manifests the reader cannot open.
- Add fal's signer "Features & Labels"; its attribution had relied on the byte
  fallback finding the "fal-ai" generator prefix.

Verdicts over 468 images and 365 videos are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-26 17:38:35 -07:00
Victor KuznetsovandClaude Opus 5.5 fb92c7dbc8 Attribute Amazon Bedrock C2PA and scope the signer fallback scan
Nova Canvas images published in aws-samples were reported as Canva: with no
Amazon row, the byte fallback matched the Canva token inside the "Nova
Canvas" software agent. Add an Amazon Web Services row ahead of Canva, which
also names Nova Reel video.

When the reader cannot open a manifest, scan for signers in the C2PA store
the container locates (JPEG APP11, PNG caBX) instead of the whole metadata
region. A vivo X300 camera capture was labeled "C2PA signer: Adobe" from its
XMP toolkit string. The SynthID provenance scan reads the same store.

Adds the Nova Canvas and Nova Reel samples as MIT-0 fixtures and records the
public-sample search for the other conformance-list signers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-26 16:21:13 -07:00
Victor KuznetsovandClaude Opus 5.5 7ba07869d2 Add paid-service captures, aggregator attribution, and provenance vetoes
Collect provider samples from Higgsfield, Runway and Dreamina plus Google
Photos, Gemini, Grok and YouTube re-encodes into data/captures/2026-09 with
a hash-checked manifest, and use them to fix attribution and false positives:

- Attribute aggregator outputs: Higgsfield job metadata, Dreamina's
  re-signed manifests, Runway's own C2PA signer, TC260 video producers.
- Veto any visible video mark when a valid C2PA claim names another vendor
  (a Gemini Omni clip read as Sora on steam), and gate Kling and Hailuo
  matches on edge, anchor and white-label evidence.
- Veto the Gemini sparkle when trusted provenance names another vendor.
- Report SynthID as present for Google Photos AI edits: Google's checker
  found it on all four measured edits, though manifests record no action.
- Add Vidu video and Wan image marks, TC260 audio placements, and record
  the AIGC key in metadata records.
- Document the research sweep, the Dreamina badge decision, and verified
  Google statements on SynthID in Photos.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-26 14:32:53 -07:00
Victor Kuznetsov d25cada76b Export calibrated image source providers 2026-09-25 14:38:04 -07:00
Victor Kuznetsov b1f15818d7 Add pinned OpenAI and Google image source classifier 2026-09-25 14:23:01 -07:00
Victor Kuznetsov 317124ee85 Keep model prereleases out of package distribution 2026-09-25 13:54:06 -07:00
Victor Kuznetsov 0d468b6022 Add verified publication path for image source model 2026-09-25 13:25:20 -07:00
Victor KuznetsovandClaude Opus 5.5 1dc70df289 Upgrade to OpenCV 5 with frozen Hershey templates and tighten video marks
- Lock OpenCV 5 and cap NumPy below 2.4 through uv constraint-dependencies so
  PaddleX 3.7 keeps resolving; Dependabot ignores mirror the cap.
- OpenCV 5 draws Hershey putText glyphs differently, so the Dola and Kling
  video templates ship as committed OpenCV 4 renders with a recipe script, and
  the OpenArt alpha recipe now refuses any OpenCV but 4.x.
- Break Sora/Veo cross-matches in favor of Veo under valid Google AI-video
  provenance, and require the full Sora mascot shape for mascot-only matches.
- Let the Doubao engine detect marks tinted by saturated backgrounds.
- Share C2PA claim-generator, credential-level and vendor-platform helpers,
  JPEG suffixes and the DCT basis instead of per-module copies.
- Log model download failures, document the optional GITHUB_TOKEN, mark the
  photo and source classifier model cards archived, and refresh the lock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:19:00 -07:00
Victor Kuznetsov 6f629a4ea6 Release 0.42.0 2026-09-22 09:25:21 -07:00
Victor KuznetsovandClaude Sonnet 5 e3bbe88155 Authenticate C2PA registry-sync GitHub API calls to fix CI rate-limiting
scripts/sync_c2pa_soft_bindings.py --check hits api.github.com to resolve
the latest upstream commit SHA, unauthenticated. That endpoint's 60/hour
rate limit is per source IP, which GitHub Actions runners exhaust fast
since many workflows share the runner IP pool -- this was failing the
lint job with "403: rate limit exceeded" on every push regardless of
whether the change touched anything related.

Send an Authorization: Bearer header built from GITHUB_TOKEN/GH_TOKEN
when set, scoped to api.github.com only (raw.githubusercontent.com is a
separate, much higher-limit CDN and doesn't need it). Wire the workflow's
own GITHUB_TOKEN secret into that step's env so CI actually has one to
send, raising the limit to 5000/hour.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 08:11:48 -07:00
Victor Kuznetsov be8f24c7cb Raise OpenAI Qwen strength floor 2026-09-21 23:02:49 -07:00
Victor KuznetsovandClaude Sonnet 5 9990f3e6ca Fix missing OpenArt gallery fixture and alternate-size entry
The OpenArt visible-mark engine (commit 9d29d2c) was registered without
a data/fixtures/visible/openart/example.png gallery canary and without
an _ALTERNATE_SIZE entry in test_visible_fill_quality.py, breaking
TestGallery::test_every_registered_mark_has_an_example,
TestGallery::test_engine_detects_its_own_example[openart], and
test_cv2_mask_quality_smoke[alternate-openart] on every CI platform.
Generate the fixture via render_visible_examples.py (self-detects at
conf=0.968) and add the missing size entry.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-21 22:37:21 -07:00
Victor KuznetsovandClaude Sonnet 5 83be3a8926 Add generic bare "AI生成" text-mark detector (brand-less TC260 fallback)
Every existing text-mark engine (Doubao, Jimeng, Qwen, Baidu, Kling,
Yuanbao) is tuned to one vendor's exact wordmark silhouette, so none of
them match a bare "AI生成" strip with no brand name in front of it.
Confirmed on raiw-app production: a free `visible` run left a bottom-right
"AI生成" watermark completely unremoved because no detector fired.

Research found the wording is not tied to one dominant generator: vivo's
own support documentation confirms its Gallery stamps a bare "AI生成"
label after AI消除/AI扩图/大片模式 editing, and independent reporting
documents the same watermark on Xiaomi Gallery's AI-edit output. Since
the label is a GB 45438-2025/TC260 compliance stamp rather than one
vendor's brand mark, this adds one generic fallback (`generic_ai_label`)
instead of another per-OEM tuned config.

The detection silhouette is font-rendered (no logo element to capture)
via the shared scripts/render_vendor_silhouettes.py pipeline already
used for Qwen/Baidu/Kling/Yuanbao's own text-only marks. Calibration is
synthetic-only (no real captured generator screenshot was available);
the module docstring and DETECT_NCC_THRESHOLD comment carry that caveat
explicitly and this should be recalibrated once real captures exist.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-21 22:01:10 -07:00
Victor KuznetsovandClaude Sonnet 5 9d29d2c777 Add OpenArt visible watermark detection and removal
OpenArt.ai stamps free-plan generations with a semi-transparent white
"OpenArt" wordmark (bowtie/infinity icon + brand name) placed near the
CENTER of the frame rather than in a corner. Confirmed from a customer-
reported raiw-app production case where the mark went completely
undetected and unremoved, with no C2PA/EXIF/TC260 signal for identify
to fall back on.

- Add a new "cc" (frame-center) corner mode to the shared
  TextMarkConfig/TextMarkEngine base, alongside the existing
  corner/edge anchors.
- Add openart_engine.py, following the existing text-mark engine
  pattern. No real capture exists (OpenArt requires an account, out of
  scope to create), so the bundled alpha asset is a procedural
  reconstruction of the public logo and the NCC threshold is an
  explicitly-labeled provisional guess, not a measured calibration.
- Register the mark in watermark_registry.py, which auto-derives
  identify's platform attribution.
- Add tests/test_openart_engine.py (synthetic composites only) plus
  registry/identify test coverage, and document the mark and its
  calibration caveats in README.md, docs/supported-signals.md,
  docs/module-internals.md, and the skill's commands.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-21 21:51:46 -07:00
Victor KuznetsovandClaude Sonnet 5 7a6491fdc1 Document Gemini sparkle decorative-glyph false positive, add observability
A decorative sparkle/glint glyph in stylized typography (e.g. a book-cover
title) can clear the visible Gemini sparkle's false-positive gate at
confidence just under _SPARKLE_FP_CONF, since it shares the same bright,
near-white, sharp four-point signature the gate uses to recognize a genuine
sparkle. Confirmed against a raiw-app production incident (2026-08-27) where
this caused the removal step to permanently smudge unrelated cover art.

Investigated tightening _SPARKLE_FP_CONF and the margin/gradient/saturation
checks (plus a prototyped local edge-density signal); each was disproved by
a constructed counter-example showing a genuine weak Gemini sparkle over a
noisy/textured photo background lands in the same confidence band with the
same signature, so no safe targeted fix exists without a real labeled
corpus. No detection/removal behavior changed.

Adds DEBUG logging at the exact FP-gate bypass path for future calibration,
a regression test (TestDecorativeGlyphFalsePositive) pinning the confirmed
false positive and the blocking counter-example, and a docs record of the
finding in the Gemini sparkle section of module-internals.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-21 21:41:59 -07:00
Victor Kuznetsov 37431c9aeb Pin Web oracle proxy sessions per batch 2026-09-16 16:17:36 -07:00
Victor Kuznetsov 804735aa48 Release 0.41.1 2026-09-16 10:42:15 -07:00
Victor Kuznetsov e3f714dcd2 Merge pull request #109 from wiltodelta/dependabot/uv/minor-and-patch-138d877835
Bump the minor-and-patch group with 6 updates
2026-09-16 09:25:05 -07:00
dependabot[bot] 7169780dd0 Bump the minor-and-patch group with 6 updates
Bumps the minor-and-patch group with 6 updates:

| Package | From | To |
| --- | --- | --- |
| [accelerate](https://github.com/huggingface/accelerate) | `1.14.0` | `1.15.0` |
| [huggingface-hub](https://github.com/huggingface/huggingface_hub) | `1.30.0` | `1.31.0` |
| [onnxruntime](https://github.com/microsoft/onnxruntime) | `1.29.0` | `1.30.0` |
| [openai](https://github.com/openai/openai-python) | `3.9.0` | `3.13.0` |
| [ruff](https://github.com/astral-sh/ruff) | `0.16.6` | `0.16.7` |
| [pyright](https://github.com/RobertCraigie/pyright-python) | `1.1.411` | `1.1.414` |


Updates `accelerate` from 1.14.0 to 1.15.0
- [Release notes](https://github.com/huggingface/accelerate/releases)
- [Commits](https://github.com/huggingface/accelerate/compare/v1.14.0...v1.15.0)

Updates `huggingface-hub` from 1.30.0 to 1.31.0
- [Release notes](https://github.com/huggingface/huggingface_hub/releases)
- [Commits](https://github.com/huggingface/huggingface_hub/compare/v1.30.0...v1.31.0)

Updates `onnxruntime` from 1.29.0 to 1.30.0
- [Release notes](https://github.com/microsoft/onnxruntime/releases)
- [Changelog](https://github.com/microsoft/onnxruntime/blob/main/docs/ReleaseNotesWorkflow.md)
- [Commits](https://github.com/microsoft/onnxruntime/compare/v1.29.0...v1.30.0)

Updates `openai` from 3.9.0 to 3.13.0
- [Release notes](https://github.com/openai/openai-python/releases)
- [Changelog](https://github.com/openai/openai-python/blob/main/CHANGELOG.md)
- [Commits](https://github.com/openai/openai-python/compare/v3.9.0...v3.13.0)

Updates `ruff` from 0.16.6 to 0.16.7
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.16.6...0.16.7)

Updates `pyright` from 1.1.411 to 1.1.414
- [Release notes](https://github.com/RobertCraigie/pyright-python/releases)
- [Commits](https://github.com/RobertCraigie/pyright-python/compare/v1.1.411...v1.1.414)

---
updated-dependencies:
- dependency-name: accelerate
  dependency-version: 1.15.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: huggingface-hub
  dependency-version: 1.31.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: onnxruntime
  dependency-version: 1.30.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: openai
  dependency-version: 3.13.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: ruff
  dependency-version: 0.16.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: pyright
  dependency-version: 1.1.414
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-09-16 08:49:01 +00:00
Victor Kuznetsov c7021b900b Add source pipeline classifier runtime 2026-09-15 23:17:48 -07:00
Victor Kuznetsov 3f55df6f99 Correct source classifier activation schema 2026-09-15 22:55:46 -07:00
Victor Kuznetsov 4795149d34 Publish source classifier model card 2026-09-15 22:36:34 -07:00
Victor Kuznetsov cf30b4f95a Release 0.40.6 2026-09-15 22:00:05 -07:00
Victor Kuznetsov 8ce8bf14da Document GPT Image 2.5 compatibility 2026-09-15 21:32:46 -07:00
Victor Kuznetsov 3fb94b220d Wait for PyPI simple index before distribution 2026-09-15 21:05:14 -07:00
Victor Kuznetsov d2c5aa66fc Merge remote-tracking branch 'origin/main' 2026-09-15 19:21:01 -07:00
Victor Kuznetsov 380820fa7e Add SDXL verified-text restoration 2026-09-15 19:20:37 -07:00
Victor Kuznetsov 12e9a87865 Publish abstaining source pipeline classifier 2026-09-15 19:18:35 -07:00
Victor Kuznetsov 19fccf27cb Release 0.40.4 with calibrated invisible profiles 2026-09-15 16:49:41 -07:00
Victor Kuznetsov a385aab433 Calibrate provider-specific invisible removal profiles 2026-09-15 16:38:17 -07:00
Victor Kuznetsov 1d7a6270d4 Add phase-free spectral pipeline probe 2026-09-15 16:20:31 -07:00
Victor Kuznetsov b5cd16aef0 Add local Microsoft InvisMark payload disruption 2026-09-14 12:07:00 -07:00
Victor Kuznetsov bf63e8e187 Attribute discovered C2PA signers 2026-09-14 11:51:19 -07:00
Victor Kuznetsov d4181f3277 Complete visible signal discovery scanner 2026-09-13 17:46:29 -07:00
Victor Kuznetsov bd14f40c7d Complete corpus signal discovery fixes 2026-09-13 16:54:58 -07:00
Victor Kuznetsov c4edc2820d Expand provenance metadata discovery 2026-09-12 13:36:46 -07:00
Victor Kuznetsov 927429d349 Fix Space pin waiter indent so distribute can find the run 2026-09-11 11:01:04 -07:00
Victor Kuznetsov df94f2c908 Release 0.40.1 with Qwen symbol false-positive fix 2026-09-11 10:55:28 -07:00
Victor Kuznetsov b2e46ee8db Automate Space pin and lengthen the ComfyUI waiter 2026-09-11 10:19:25 -07:00
Victor Kuznetsov b03a65bcb3 Release 0.40.0 with Qwen symbol, ONNX classify, identify attribution 2026-09-10 23:04:58 -07:00
Victor Kuznetsov f2f840197e Stop attributing shared AI labels to vendors 2026-09-10 21:08:18 -07:00
Victor Kuznetsov 242d78957b Pin published ONNX classifier snapshot 2026-09-10 20:07:33 -07:00
Victor Kuznetsov 78426aeef7 Add FP32 ONNX photo classifier backend 2026-09-10 19:31:24 -07:00
Victor Kuznetsov d914e3ac08 Add comprehensive signal discovery workflow 2026-09-10 17:40:10 -07:00
Victor Kuznetsov c903346a6b Support Qwen Create symbol watermark 2026-09-10 17:36:26 -07:00
Victor Kuznetsov 7cf067d3a5 Separate TC260 marks by manufacturer 2026-09-10 17:11:26 -07:00
Victor Kuznetsov 15cae6fc91 Speed up CLIP classifier cold start 2026-09-10 16:45:04 -07:00
Victor Kuznetsov d3a042209c Align maintenance gate and enable project plugins 2026-09-10 10:00:24 -07:00
Victor Kuznetsov 1386708dec Add manual skill catalog updates to release flow 2026-09-10 09:37:20 -07:00
Victor Kuznetsov 67665c5cea Record agent skill directory submissions 2026-09-09 21:32:50 -07:00
Victor Kuznetsov fc54f25948 Add concrete examples to agent skill 2026-09-09 20:11:46 -07:00
Victor Kuznetsov f020115ca2 Release 0.39.1 2026-09-09 18:51:48 -07:00
Victor Kuznetsov 62be1ef456 Drop third-party apt sources before the video job's ffmpeg install
The video full-clip job needs ffmpeg from Ubuntu's own repositories, but its
apt-get update refreshed every source on the runner image, so a broken
third-party index failed the job twice on 2026-09-09: google-chrome.list
served a Hash Sum mismatch (bc1428ab... against 233e56de...) and both the run
and its rerun died at Install ffmpeg with ten otherwise-green jobs around
them. Remove every non-Ubuntu source definition from
/etc/apt/sources.list.d before updating, so no vendor's broken index can
fail a step that never needed it. The only apt-get in the workflows.

pre-commit: 1) maintain.sh - exit 2 at the uv-secure gate, the three triaged
no-fix advisories in docs/development.md (rechecked 2026-09-09 against PyPI);
ruff check+format clean, pyright src/ 0 errors, 2368 passed 3 skipped run
separately, same gate run as the preceding commit with no post-gate edit
between them; 2) /simplify - single pass, no fan-out available, no findings;
3) docs sync - grepped apt-get/google-chrome/Install ffmpeg across docs/,
README, .claude/rules/, tests/: no other references, the rationale lives in
the step's comment; 4) CLAUDE.md - audited against the structure rule, no
additions, revise-claude-md: no additions
2026-09-09 18:27:22 -07:00
Victor Kuznetsov 4d7c2b458e Record MIT-0 as ClawHub's platform constant, not a listing error
The listing's MIT-0 next to the repository's Apache-2.0 was documented as a
drift to repair in the ClawHub listing itself. It is not repairable and not
drift: ClawHub's own client pins the platform skill license
(dist/schema/license.js sets PLATFORM_SKILL_LICENSE = "MIT-0", and the
version schema accepts only "MIT-0" or null), so every skill published there
is offered under MIT No Attribution. Publishing to ClawHub means offering
this skill on those terms in addition to Apache-2.0; the field is not
editable and the mismatch must not be read as drift.

pre-commit: 1) maintain.sh - exit 2 at the uv-secure gate, the three triaged
no-fix advisories in docs/development.md (rechecked 2026-09-09 against PyPI);
ruff check+format clean, pyright src/ 0 errors, 2368 passed 3 skipped run
separately; 2) /simplify - single pass, no fan-out available, prose-only
change, no findings; 3) docs sync - MIT-0/ClawHub grepped across docs/,
README, .claude/rules/, skills/: only release-and-distribution.md describes
the publish flow and stays accurate, the audit doc row is a dated snapshot,
no refs to update; 4) CLAUDE.md - audited against the structure rule, no
additions, revise-claude-md: no additions
2026-09-09 18:27:08 -07:00
Victor Kuznetsov 96b8ab9ae2 Let the HEIF container, not the blanked tag, decide the orientation carry
pillow-heif's opener rewrites the EXIF orientation to 1 and stashes the real
value in info["original_orientation"], while libheif turns the raster on decode
exactly when the container declares an effective irot/imir transform. A camera
HEIC (stored raster, EXIF-only rotation) and a re-encoded one (rotation baked
as irot) both decode with Image.size matching the raster and the tag blanked,
so _read_display_tags dropped the turn for both (issue #105): a camera HEIC
went through erase and came out sideways with nothing on it saying so.

For HEIF the caller's orientation_applied claim is now ignored the way it is
for TIFF, and the container decides: isobmff.heif_transform_applied scans
meta/iprp/ipco for an irot with a non-zero angle or any imir, the orientation
comes from original_orientation, and a transform-bearing file drops the tag.
AVIF keeps the generic path: this plugin build neither blanks its tag nor
applies the transform there, and a camera-style AVIF's orientation stays
unreachable from Pillow (recorded in known-limitations).

Tests build the camera fixture without exiftool: a pillow-heif-written file
with its irot angle neutralized to the identity behaves like the reporter's
exiftool-built camera file. Mutation-checked in both directions: forcing the
detector either way fails exactly the tests whose property broke.

pre-commit: 1) maintain.sh - exit 2 at the uv-secure gate, the three triaged
no-fix advisories in docs/development.md (rechecked 2026-09-09: accelerate
1.15.0 and lightning 2.6.5 still carry no fix); ruff check+format clean,
pyright src/ 0 errors, 2368 passed 3 skipped run separately; 2) /simplify -
single pass, no fan-out available, one duplicate test helper inlined, no
other findings; 3) docs sync - heif_transform_applied, original_orientation,
irot, imir live only in module-internals.md and known-limitations.md, both
updated, no other refs; 4) CLAUDE.md - audited against the structure rule,
no additions; revise-claude-md: no CLAUDE.md changes
2026-09-09 18:02:05 -07:00
Victor Kuznetsov 5b30f0169a Merge pull request #107 from DKalimist/fix/heif-and-tiff-orientation
Read a TIFF source's stored size from its IFD, not from Image.size
2026-09-09 17:16:34 -07:00
Victor Kuznetsov 003dabfae4 Let the TIFF geometry override a caller's raw-decode claim
Both readers in the imread stack apply the TIFF orientation tag (cv2's
libtiff path under IMREAD_UNCHANGED too, and Pillow's TIFF plugin,
identically), so the visible path's blanket orientation_applied=False
carried the tag onto an already-turned raster. _read_display_tags now
ignores the explicit claim for TIFF and decides on the IFD's stored size,
which also keeps a hypothetical non-turning build correct: a stored-shape
raster still carries the tag. Guarded at the api seam, not just imwrite.

pre-commit: 1) maintain.sh - exit 2 at the uv-secure gate, the three
triaged no-fix advisories in docs/development.md (rechecked 2026-09-08);
ruff check+format clean, pyright src/ 0 errors, 2363 passed 3 skipped
run separately; 2) /simplify - single pass, no fan-out available, nothing
to fix (api-state plumbing rejected as disproportionate); 3) docs sync -
identifiers live only in docs/module-internals.md, updated, no other
refs; 4) CLAUDE.md - audited against the structure rule, no additions,
revise-claude-md: no additions
2026-09-09 12:56:14 -07:00
Victor Kuznetsov 4c090c5ae2 Merge main into fix/heif-and-tiff-orientation
Resolve image_io.py and test_image_io.py the same way the rebased
adaptation branch does, so the branch carries the author's original
commit 1cfdf19 on top of current main.
2026-09-09 12:56:14 -07:00
Victor KuznetsovandClaude Opus 5 8c025985f7 Correct the ClawHub issue state to closed
The previous commit recorded openclaw/clawhub#3643 as still open. It was
closed as completed at 17:29 UTC, minutes before that commit landed, so the
claim was already false when it was written: the state was read earlier in
the session and never re-read against the tracker.

The note now says the reporter closed it once 1.0.5 published, that nothing
upstream was confirmed fixed, and that this was the fifth report of the same
shape, so a future stalled submission gets a fresh report rather than an
assumption that the pipeline is repaired.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PrJja6MnncVhyWnmXPN4GM
2026-09-09 10:38:10 -07:00
Victor KuznetsovandClaude Opus 5 5f1dea95f4 Record the ClawHub listing as live and correct its blocker note
docs/agent-skill.md still described the ClawHub surface as blocked: 1.0.3
and 1.0.4 stuck in pending.publication, with an instruction not to bump the
skill version until openclaw/clawhub#3643 resolved. The registry disagrees.
1.0.5, submitted by the clawhub job during the 0.39.0 release, is public:
clawhub search returns the row, and the public API reports tags.latest
1.0.5 with a single published version, which is also how the two earlier
submissions are shown never to have gone public. They were bypassed by the
next submission rather than recovered, and the pipeline issue stays open.

The note now says to read the catalog rather than the job log, since a
green job proves only a submission. It also records that the job fires on a
published release alone, so the 1.0.6 bump that landed after 0.39.0 reaches
ClawHub with the next release and must not be published by hand, and that
the listing carries MIT-0 where SKILL.md declares Apache-2.0, which the
publish command has no flag to set.

The awesome-openclaw entry loses its ClawHub prerequisite and stays
deferred on adoption alone: the listing reports no installs in 60 days.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PrJja6MnncVhyWnmXPN4GM
2026-09-09 10:31:56 -07:00
Victor Kuznetsov 00d9771c6a Raise the skill's CLI floor to cover its 0.39.0 references
commands.md names liblib_pill and the auto pipeline routing OpenAI
provenance to qwen-zimage, both of which only exist since CLI 0.39.0,
while probe.py kept MIN_CLI_VERSION at 0.38.0: a 0.38.0 install would
pass the probe's currency check and then reject the very mark key and
routing the skill hands it. The references now carry the established
(since CLI X.Y.Z) annotations and the floor rises to 0.39.0, per
docs/agent-skill.md.

The floor was hand-synced against prose only a person could read, so it
drifted exactly the way its own comment warns about, twice before. A new
guard derives the floor's lower bound from every (since CLI X.Y.Z)
annotation in the skill tree and fails when the floor lags the newest
one; combined with the existing package-version ceiling, the floor is
now pinned from both sides. Skill version 1.0.6 in SKILL.md, plugin.json,
and marketplace.json.

pre-commit: 1) maintain.sh - exit 2 (uv-secure on the documented
lightning/accelerate blocks, unchanged lock; core checks separately:
C2PA sync current, Ruff 1 fix + 1 reformat folded into this commit,
Pyright src/ 0 errors, 2361 passed 3 skipped); 2) /simplify -
single-pass four angles, clean; 3) docs sync - grepped MIN_CLI_VERSION
and since CLI over docs/ and rules/: both mentions stay accurate;
4) CLAUDE.md - improver grade A earlier this session, revise: no
additions
2026-09-08 21:31:25 -07:00
Victor Kuznetsov de03748849 Read the flat single-artifact layout in release verification
download-artifact@v8 extracts the only pattern-matched artifact directly
into the target path, without the artifact-name directory the resolve
step assumed, so every workflow_run verification died at
"No distributed release identity found" before checking a single
surface. The directory-name attempt check now applies only when the
artifact directory exists; a flat identity is still pinned by the
run-id and attempt-eligibility asserts. The workflow_run path of
5c6588b had never executed: v0.38.0's run failed on later checks and
the surfaces were confirmed by a manual dispatch.

The new test drives the real workflow script against the flat layout
and reproduced the production failure before the fix.

pre-commit: 1) maintain.sh - exit 2 (uv-secure on the documented
lightning/accelerate blocks, unchanged lock; core checks separately:
C2PA sync current, Ruff clean with no rewrites, Pyright src/ 0 errors,
2360 passed 3 skipped); 2) /simplify - single-pass four angles, clean;
3) docs sync - grepped distributed-release over docs/: the one mention
stays accurate; 4) CLAUDE.md - improver grade A earlier this session,
revise: no additions
2026-09-08 20:30:59 -07:00
Victor Kuznetsov e556ed032a Release 0.39.0 with generator taxonomy and the LiblibAI pill
pre-commit: 1) maintain.sh - exit 2 (uv-secure on the documented
lightning PYSEC-2026-3624/GHSA-qqmf-gpg7-g8gw and accelerate
GHSA-4j2p-28q2-5m79 blocks, no fixes on PyPI; core checks separately:
C2PA sync current, Ruff clean with no rewrites, Pyright src/ 0 errors,
2359 passed 3 skipped); 2) /simplify - single-pass four angles, clean;
3) docs sync - grepped 0.38.0/0.39.0 over README, docs/, skills/,
rules/: no stale refs (0.38.0 hits are feature-first mentions or the
diffusers dependency); 4) CLAUDE.md - improver grade A, no changes;
revise: no additions
2026-09-08 20:14:35 -07:00
Victor Kuznetsov 557f776564 Record the ClawHub pending-publication blocker and support issue 2026-09-08 20:13:49 -07:00
Victor Kuznetsov 36fed0594e Fix sharpening quantization and cross-platform regression tests
Round unsharp output in its existing buffer before uint8 conversion, preserving
constant images across OpenCV builds. Compare explicit stored WebP decodes.
Use host-absolute launcher fixtures and restrict Ubuntu Bash harnesses to POSIX.

Document the CI findings and the unresolved Dependabot Transformers/tokenizers
candidate conflict. Keep the existing model dependency constraints.
2026-09-08 19:25:44 -07:00
Victor Kuznetsov 5c6588b3ef Fix output preservation, research validation, and documentation drift
Repair image and metadata publication contracts, detection routing, research
artifact validation, release verification, and the published agent skill.
Refresh the dependency lock and reconcile documentation with verified behavior.

Add regression coverage and record the completed audit and corrected VideoSeal
measurements. Preserve the existing Accelerate and Lightning versions.

Validation: Ruff and Pyright pass; 2358 tests pass with 3 skipped. The maintenance
gate exits 2 on the documented Accelerate and Lightning security findings.
2026-09-08 19:07:10 -07:00
Victor Kuznetsov f127ecbe57 Bump the agent skill to 1.0.4 to re-queue ClawHub scans
The ClawHub publication of 1.0.3 has sat in pending.publication with
succeeded static scans and null primary verdicts; version numbers stay
reserved, so the only re-queue path is a new version. No content change.
2026-09-08 14:41:00 -07:00
Victor Kuznetsov 56afcab241 Merge feat/model-freeze: generator taxonomy, model-side receipt gate
bytedance generator class (surgical six-head provider.pt), tc260 group
veto, stable-name receipt-gate.npz with the threshold inside the
artifact, snapshot-tolerant loader, weights pinned to revision
4c276376 (published via photo-classify-freeze-2026-09-07 release).
2026-09-08 14:18:04 -07:00
Victor Kuznetsov 4428aedce6 Prefer provider APIs before web oracles 2026-09-08 14:02:25 -07:00
Victor Kuznetsov 3daca46589 Detect and remove LiblibAI compact top-left pill 2026-09-08 13:37:07 -07:00
Victor Kuznetsov 15f5fee7c3 Add unified provider oracle tooling 2026-09-08 13:26:43 -07:00
Victor KuznetsovandClaude Opus 5 3ee858e195 Keep the boundary rule from naming what it protects
The ignore rule landed with a comment naming the private product, its storage
provider, the private rule file, and the size of the customer set. That text is
the disclosure the rule exists to prevent, so it says what a public reader needs
instead: these trees are local-only and not publication-cleared.

`data/research/` gets the same tracked rule as `data/spaces/`. It held 68 GB
behind `.git/info/exclude` alone, which no clone has, so a fresh checkout could
commit the whole corpus with nothing to stop it, and a test now pins both
entries.

A first draft of that test also carried a denylist of private identifiers,
which published six of them into a tracked file to keep two out, and only its
own self-exclusion hid that. The mechanized half is now the one that discloses
nothing: no cloud GPU harness under `scripts/`. Reading the ignore entries
rather than the file text matters too, because a comment naming
`data/research/README.md` satisfied the first assertion with the rule deleted.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-08 10:43:20 -07:00
Victor KuznetsovandClaude Fable 5.1 cd225f2af9 Keep customer uploads out of this public repository by rule, not by luck
data/spaces holds 157k real user images that the private raiw-app retention
checks materialize here. The only thing keeping them out of a commit was an
untracked .gitignore inside the directory, which a fresh clone does not
have.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 10:09:11 -07:00
Victor KuznetsovandClaude Opus 5 e85307f1c0 Scrub private infrastructure detail and move the GPU harnesses out
The Content Seal manifest named the production GPU worker in its provenance
column and in two row keys, several docstrings described their sources as user
uploads from the private corpus, and that corpus was called by the storage
product holding it. Each is now what a public reader can check: a pipeline and
strength, material that is not publication-cleared, and a private corpus. The
corpus test follows the renamed rows. Deployment and tier detail goes the same
way, since which backend each hosting tier can afford is a product decision,
not a library fact.

Twelve harnesses that need a cloud GPU account leave scripts/. The research
pages that cite their findings name them as harnesses kept outside this
repository and stop implying a reader can rerun them; the tracked inputs and
recorded verdicts remain the durable record, and the offline analysis step
stays here with its test. A development invariant states the rule so the next
harness is not added back.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-08 09:29:12 -07:00
Victor Kuznetsov 5a875daded Update the pinned-revision test for the 2026-09-07 snapshot
The previous commit slipped past a red test: the pytest exit code was
swallowed by a pipe in the shell chain. This pins the test to
4c2763766dd1c8c212d64e01e1cc3f166243c47d.
2026-09-07 21:57:54 -07:00
Victor Kuznetsov c9672efd46 Pin classify weights to the 2026-09-07 snapshot revision
Published via publish-photo-classify-hf.yml (run 34188279796) from the
photo-classify-freeze-2026-09-07 release: six-head provider.pt, model-side
receipt-gate.npz, updated card and operating point. Older lib versions
keep their own pinned revisions byte-for-byte.
2026-09-07 21:57:11 -07:00
Victor Kuznetsov 8562c457a4 Ship generator taxonomy: bytedance class, model-side receipt gate
Provider classes now name the GENERATOR that rendered the pixels, never
the brand. bytedance joins openai/google/muse-image as the shared
ByteDance lineage (Doubao and Jimeng render with one model family;
measured: separate heads cross-fire 32/22 at 3.5x mass, the surgical
union holds 83.9% on the frozen test cell while openai/google/meta/no_ai
stay byte-identical to run1). tc260 stops being a public value: its head
remains as the group veto for the rest of China's generators (without
it, 207/379 China test rows would be falsely named openai or
muse-image), and the loader tolerates provider snapshots that predate
bytedance. The receipt gate moves model-side: stable-name artifact
receipt-gate.npz carries its threshold, head and threshold always load
from one file, and the publish script stages the stable gate from the
weights source. Microsoft stays metadata-only: 144 of its 253 bank rows
name an OpenAI model as their C2PA softwareAgent and none names a
Microsoft renderer.

pre-commit: 1) maintain.sh - exit 1 (uv-secure: lightning 2.6.5 via
trustmark, PYSEC-2026-3624 no fix released; per docs/development.md the
core checks ran separately: ruff clean, pyright 0 errors, 2059 passed
3 skipped); 2) /simplify - single-pass four angles, clean, one noted
skip (gate-test npz staging duplication); 3) docs sync - grepped
bytedance/receipt-gate.npz/provider values over README, docs, skill,
rules: no stale references; 4) CLAUDE.md - improver: no changes,
revise: no additions (learnings live in canonical docs)
2026-09-07 21:13:48 -07:00
Victor KuznetsovandClaude Fable 5.1 17c034f140 Scope the development rule by paths and point at the security-gate triage
The rule used globs, which Claude Code ignores, so it loaded in every
session. CLAUDE.md keeps the fact that this gate does not pass
--ignore-unfixed and routes the lightning triage to docs/development.md.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-07 19:37:49 -07:00
Victor KuznetsovandClaude Opus 5 5b12f61db8 Record why the OpenAI chroma floor stays flat
Three withheld zero-face carriers, all confirmed zero-face with the same YuNet
detector the adaptive arms gate on, first cleared at 0.045 / 0.06 / 0.11. The
zero-face operating point is 0.11 + (0.11 - 0.045) = 0.175, which the shipped
0.20 already covers, so that floor now rests on both content classes rather than
only the face-bearing carriers it was derived from.

The content split this cohort kept suggesting does not survive the measurement:
the two classes differ by 0.025 while the spread WITHIN each is 0.065 and
0.0625, so a face split would key on a variable explaining about a quarter of
the variation it leaves behind.

The methodological point is the durable half. The fixture-only view of zero-face
OpenAI content spans 0.015 of boundary; the real carriers span 0.065, four times
wider. A narrow fixture spread yields a small worst-clean-plus-one-spread margin
and a false impression of homogeneity, and that impression is what made a split
look justified twice.

Every verdict is the public verifier's separate SynthID line with Content
Credentials reported not detected alongside, over RGB-identical metadata-stripped
positive controls.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-07 19:08:08 -07:00
Victor Kuznetsov ee19173338 Record the ETI blur+JPEG protocol filter as a conditional follow-up
The ETI scoring program applies median blur and JPEG before decoding, so an
attack only wins if the watermark stays dead under ordinary handling. Our
removal paths are structural, so no driving question exists today; the note
records the hour-sized handling arm over removed-state artifacts as the
follow-up if removal durability under platform re-encoding ever becomes a
real question. Also records that participant attack code was never
published.
2026-09-07 18:23:19 -07:00
Victor Kuznetsov 98d3df9f5a Record the ETI and W-Bench licensing verdict
Verified against primary sources: the ETI Competition Data (NeurIPS 2024)
carries no license on the dataset card or in the evaluation repository and
stays off limits without organizer permission; the W-Bench dataset (ICLR
2025) is MIT while the VINE code is NTUITIVE non-commercial, so a profile
is buildable on our kernel without the VINE code. Also corrects two stale
internal notes: ETI attacked outputs are in the release, and W-Bench holds
10,000 instances, not 15k.
2026-09-07 18:23:18 -07:00
Victor Kuznetsov 464fd142ea Add audio and video watermark benchmark oracles, cohorts, and research studies
Extend the development-only benchmark family from images to audio and
video, and close four research directions from the watermark-landscape
map with local, revision-pinned evidence.

Benchmark schema: media_type audio and video rows, audioseal and
videoseal adapters with media-checked adapter binding, sample-domain
audio fidelity (SNR with unbounded_identical), frame-domain video
fidelity (mean PSNR, changed-frame fraction), and per-frame temporal
evidence in videoseal detection records. The report accepts all three
media types and carries SNR columns.

Audio: shared pinned AudioSeal oracle, synthetic and system-voice
speech cohorts with matched negatives, codec, noise, removed, and
forged arms, and the provenance experiment proving visual-only video
cleaning leaves the audio bitstream and watermark verdict invariant.

Video: TorchScript-only VideoSeal oracle (the pip package is unusable
on macOS arm64 and its loader resolves paths relative to cwd), video
cohort with crf/scale/fps attacks and four-state arms, temporal
evaluation measuring the crf decay curve, the frame-aggregation matrix
(computed locally after finding the TorchScript build ignores the
aggregation argument), and real provider clips as carriers.

Forgery: double-embedding study showing the last writer wins completely
under both oracles, scoped remover-trace probe, and the forged/
removed cohort arms encoding each scheme's correct expected verdict
(presence rule for audioseal, message rule for videoseal).

C2PA resolution: measured the five vendors' resolver contracts and the
C2PA 2.2 18.10.5 privacy gradient; product decision recorded that this
tool never resolves over the network. A registry guard pins the
published resolution-API set.

Gate: ruff and pyright src/ clean, 2114 passed / 3 skipped; maintain.sh
stops only on the known unfixed lightning 2.6.5 CVEs, pre-existing.
2026-09-07 18:23:18 -07:00
Diego.F 1cfdf192d8 Read a TIFF source's stored size from its IFD, not from Image.size
`_read_display_tags` drops the orientation when the raster no longer has the
source's stored dimensions, which is what stops an upright decode from being
tagged into a second rotation. It read those dimensions off `Image.size`.

That is not the stored size for TIFF: Pillow's TIFF reader applies the EXIF
orientation itself and reports the upright one. For a transposing orientation
the comparison was upright-against-upright, so it matched and the tag rode
along onto pixels that were already turned. `erase` on a 96x64 TIFF tagged 6
returned a 64x96 PNG carrying Orientation 6, and a viewer turned it again.

The TIFF IFD keeps the real geometry in ImageWidth/ImageLength, and no other
Pillow plugin exposes `tag_v2`, so every other container falls back to
`Image.size` and is byte-for-byte unaffected.

Also corrects `_pil_read`'s "No EXIF auto-rotation, matching cv2": Pillow's
TIFF reader rotates and cannot be told not to. Coming back upright is the
useful outcome; carrying the tag on top of it was the defect.
2026-09-08 00:54:22 +02:00
Victor Kuznetsov b2496134c2 Add margin to the explicit Chroma OpenAI floor 2026-09-07 14:15:57 -07:00
Victor Kuznetsov 723f2949ec Route OpenAI auto removal through Qwen 2026-09-07 14:01:17 -07:00
Victor Kuznetsov e645fac166 Record the Claude marketplace submission in the agent skill guide 2026-09-07 12:55:07 -07:00
Victor Kuznetsov 4077db8b57 Stub huggingface_hub in the allow-patterns test instead of importing it
The test monkeypatched huggingface_hub.snapshot_download by string, which
imports the real module. huggingface-hub rides the classify extra, not
the dev sync, so the CI matrix failed with ModuleNotFoundError on every
platform while the shared checkout venv hid the gap. Insert a stub into
sys.modules instead: the seam under test is classify's request, not the
hub client.
2026-09-07 11:48:37 -07:00
Victor Kuznetsov 8eec0e5af4 Publish the agent skill to ClawHub from the release fan-out
The clawhub job in distribute.yml compares the SKILL.md version with
the published one on every GitHub Release and submits an update when
they differ. Gated on the CLAWHUB_TOKEN secret: without it the job
skips with a notice, so the workflow lands safely before the secret
exists. A submitted update waits for ClawHub security scans, so a
green job means submitted, not listed.
2026-09-07 11:34:13 -07:00
Victor Kuznetsov 7ddd2097bd Export WEIGHTS_ALLOW_PATTERNS for offline pre-caches
The runtime's _weights_dir resolves the Hub snapshot offline through an
inline allow_patterns list, so a deployment that pre-caches the snapshot
for HF_HUB_OFFLINE=1 had to re-list the files by hand and silently
drifted when a release added one: the offline resolution then failed on
a four-fifths-present cache even though every missing piece had its own
fallback. Export the exact runtime request list as one public name, use
it in _weights_dir, and pin the seam with a test that captures the
snapshot_download call. Document the contract in photo-classify.md and
module-internals.md.
2026-09-07 11:31:08 -07:00
Victor Kuznetsov c0fec5de19 Release 0.38.0 with the receipt gate for classify 2026-09-06 21:16:37 -07:00
Victor Kuznetsov bf7f556876 Fix nondeterministic ICC profile assertions 2026-09-06 20:51:00 -07:00
Victor Kuznetsov f89975355d Remove retired verifier from agent skill 2026-09-06 20:27:19 -07:00
Victor Kuznetsov 68b56ece37 Add stratified watermark benchmark and resource profiling 2026-09-06 20:18:48 -07:00
Victor Kuznetsov dc8491a6c6 Add reproducible watermark cohort, timing, and reports 2026-09-06 20:18:48 -07:00
Victor Kuznetsov 4e9145f739 Add reproducible watermark benchmark kernel 2026-09-06 20:18:48 -07:00
Victor Kuznetsov 0d6667cf22 Separate runtime actions from development provenance checks 2026-09-06 20:18:47 -07:00
Victor Kuznetsov 646f72534b Carry the ICC profile and EXIF orientation across re-encodes
Issue #98. cv2's encoders write no container metadata, so every pixel-path
output (all, visible, invisible, erase) came back without its colour profile
and orientation tag, even with --keep-metadata.

imwrite now takes display_tags_from, the file whose decode produced the
pixels, and restores the two display tags into the re-encoded output: PNG and
JPEG by splicing iCCP/eXIf chunks and APP1/APP2 segments into the encoded
bytes, WebP by a lossless Pillow re-save (libwebp hides hand-spliced chunks
from readers without a VP8X header), HEIF through the save call itself.
Orientation is re-tagged only when the raster still has the source's stored
dimensions: cv2 rotates JPEG and (build-dependent) PNG decodes under
IMREAD_COLOR but never IMREAD_UNCHANGED, which is what the pixel paths read
with, so the decision is made against the raster's own shape and cannot
double-rotate.

The default metadata strip now preserves what the restore put back: the PIL
path forwards the profile and a scrubbed EXIF (orientation included) for PNG
and WebP outputs, the 16-bit chunk walker re-emits eXIf scrubbed instead of
dropping it wholesale, and --remove-all actually drops the ICC profile on
every format (Pillow's saver fallback kept it alive, contradicting the
documented "removed by --remove-all").

Mirror and 180-degree orientations (2, 3, 4) stay untagged: no cheap signal
distinguishes a rotated decode from a stored one for them.
2026-09-06 14:41:19 -07:00
Victor Kuznetsov 922f6faa7a Record the awesome-list maturity bars in the agent skill guide
ComposioHQ accepted a submission with no adoption bar (PR 1841);
VoltAgent lists reject brand-new skills, and the openclaw list indexes
ClawHub entries only. Both re-attempts are deferred with their
re-entry criteria.
2026-09-06 14:37:40 -07:00
Victor Kuznetsov f0b425ac71 Add skill packaging invariants and a path-filtered skill workflow
Three invariants the deterministic suite did not pin: the version
triple (SKILL.md, plugin.json, marketplace.json), the 500-line SKILL.md
budget, and probe.py MIN_CLI_VERSION never exceeding the package
version. skill.yml gives skill-only changes a seconds-fast gate on top
of the full Tests workflow.
2026-09-06 12:39:22 -07:00
Victor Kuznetsov 8ba2666a30 Document the text-restoration extra in the agent skill install table
CLI-reachable through invisible --text-manifest, present since v0.27.0,
so no MIN_CLI_VERSION change. Skill version 1.0.2 across SKILL.md,
plugin.json, and marketplace.json. The receipt-abstention wording for
classify is deferred to the 0.38.0 release change, where it belongs with
the MIN_CLI_VERSION raise.
2026-09-06 12:11:55 -07:00
Victor Kuznetsov c1bf835e2f Fix Kling video detection across decoders 2026-09-05 13:43:00 -07:00
Victor Kuznetsov efdbfa0598 Add real watermark fixtures and expand provider coverage 2026-09-05 13:12:58 -07:00
Diego 9f7b707016 Preserve 16-bit images through fill and metadata stripping (#101)
Keep alpha bit-exact, make remove-all drop iCCP, and bound MI-GAN conversion and PNG chunk-copy overhead.
2026-09-05 13:11:16 -07:00
Diego 697e11f2ab Report unsupported global CPU offload profiles (#104)
Handle unresolved auto preload through its Qwen fallback and avoid a false warning.
2026-09-05 12:30:00 -07:00
Victor Kuznetsov efeec825c1 Document the CI concurrency cancel behavior in development docs 2026-09-05 12:21:34 -07:00
Diego 507d7c43a3 Run adaptive polish before the humanizer (#102)
Correct the postprocessing order documentation and simplify its regression coverage.
2026-09-05 12:10:17 -07:00
Victor Kuznetsov a1022d33c3 Pin the photo-classify snapshot at the gate revision
ffc46db5 carries receipt-gate-2026-09-02.npz and the receipt_gate
operating-point entry, so fresh classify installs load the gate from the
model snapshot; the package asset remains the fallback for pre-gate
weights directories.
2026-09-05 12:08:08 -07:00
Victor Kuznetsov 5b52782f01 Move the receipt gate to the model side
The gate head is fitted on the freeze CLIP-L-ft embedding space, so it
belongs with the model: _load_receipt_gate now prefers
receipt-gate-2026-09-02.npz from the Hub snapshot or the
RAIW_CLASSIFY_WEIGHTS directory, with the package asset kept as the
fallback for weights frozen before the gate existed. Card mode of the
publish workflow now uploads the gate alongside the README and the
operating point, and the snapshot allow-pattern includes it.
2026-09-05 11:51:02 -07:00
Victor Kuznetsov 0761937d60 Improve Samsung detector recall on textured images 2026-09-05 11:51:02 -07:00
Victor Kuznetsov b8a8c3d3be Improve visible-mark evaluation and Samsung masking 2026-09-05 11:51:02 -07:00
Victor Kuznetsov 0fd139aac4 Fix agent skill dependency checks and metadata guidance 2026-09-05 11:32:22 -07:00
Victor KuznetsovandClaude Opus 5 ecbe296b31 Keep the skill-eval harness tests portable on Windows
Windows has no execute bit, so asserting one on the sandbox CLI copy failed both
Windows jobs while every other platform passed. The shim is launched through the
interpreter there anyway, so the assertion is POSIX-only.

The subprocess env is now overlaid on the real one rather than replacing it.
That was not the failure -- the hand-built env happened to work -- but a PATH of
`/usr/bin:/bin` and no SystemRoot is not something to leave in a test that CI
runs on Windows.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-05 11:27:17 -07:00
Victor KuznetsovandClaude Opus 5 09960676fc Guard the skill against the CLI, and measure it against a model
The packaging tests a catalog runs say nothing about content, so the skill's
prose drifted from the CLI exactly the way an unguarded seam does. These tests
read the live Click tree: mark keys, flag choices, batch modes, pipeline
profiles, vendor cohorts, exit codes and extras must match what the skill says.

Guard the SET, not the instances. Per-flag comparisons only check what the skill
already documents, so `--vendor`, `--pipeline auto`, and then the whole
`classify` and `verify-openai-synthid` commands each shipped with a green suite.
`test_every_cli_command_is_named_somewhere_in_the_skill` closes that.

None of it proves the only thing a skill is for: that a model reaches for it and
then obeys it. `scripts/skill_eval.py` builds a throwaway project with the skill
installed and a recording stand-in for the CLI on PATH, runs a headless agent on
one user prompt, and grades the recorded command trace. Grading is mechanical --
"did it run `all` where the probe said nothing is written" is a fact in the
trace, and "did it name an output file that is not on disk" catches a fabricated
result without asking a model to judge itself. Agents are nondeterministic, so
the report is a pass rate per check over repeats, never one verdict.

The harness is tested too: the stand-in's exit codes are compared against the
real CLI where the state is reproducible locally, the grader runs over hand-built
traces, and every case is checked for internal consistency, because a scenario
whose `identify` names a mark its `visible` cannot find measures the case rather
than the skill.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-05 11:27:17 -07:00
Victor KuznetsovandClaude Opus 5 2bc37738d9 Publish the agent skill that drives this CLI
`skills/remove-ai-watermarks/` teaches a coding agent to drive the CLI on
content the user generated, and refuses stock-agency and other third-party
paid-asset marks. It ships with this repository rather than as a fifth release
surface, and installs through `npx skills add` in any harness or as a Claude
plugin from `.claude-plugin/marketplace.json`.

`scripts/probe.py` measures capability rather than presence: a found binary said
nothing about whether the build can run a visible-mark command, so it writes a
blank PNG with zlib and struct and runs the installed `visible` and `invisible`
on it. Exit 2 is a state only a working stack reaches, and `invisible` checks
its extra before detection, so a signal-free image answers in 0.23s with no
model download. That distinction matters: without CUDA, `all` writes nothing
with the invisible extra installed and writes a useful visible+metadata result
without it, and both exit 1.

The skill also carries what a weak model gets wrong. It must report only what a
command in the session actually did, because a run on a machine with no pixel
stack produced a finished-looking removal report, complete with a size table for
a file nobody wrote. It must not read exit 2 as a verdict, because Click's usage
errors share that code. And it must get explicit consent before
`verify-openai-synthid`, which uploads the user's image to a third party.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-05 11:27:17 -07:00
Victor KuznetsovandClaude Opus 5 637395b8a6 Name the missing pixel stack instead of tracebacking on it
The default package installs without `pixels`, and the Homebrew formula ships
exactly that build, so a first run that followed this project's own install
instructions died mid-command on a bare `ModuleNotFoundError: No module named
'cv2'` -- the one moment the user needed the extra named, and the only moment
they were not told it. `visible`, `erase`, `all` and `batch` now stop with the
install hint; `remove_batch` lets the same error out instead of counting one
absent package as an error per image, which turned a directory of 84 into 84
failures with no cause named.

Ask `optional_deps.pixels_available()` rather than matching the exception's
module name. An ImportError raised inside `cv2/__init__.py` carries `cv2.cv2`
and a numpy ABI mismatch carries `None`, so a name match let exactly the
tracebacks this guard exists for through.

`identify` keeps its historical no-op when the visible arm cannot decode, but
the report no longer stays silent about it: an install lacking the extra dropped
the visible Gemini mark from a marked image and read exactly like a clean scan.
The caveat names which silence happened, because an absent extra and an
unreadable file need different fixes.

`metadata --remove` reports surviving markers instead of crashing when the
raster-normalization recovery cannot import cv2, and the video runtime's install
hint is now shell-quoted like the others -- bare brackets are a glob in zsh.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-05 11:27:16 -07:00
Victor Kuznetsov 1e71f0523d Document the provider drift retrain campaign
The 2026-09-02 campaign measured the Aug ChatGPT renderer drift at the
Model 2 level and shipped nothing: leave-one-over-one recovers only the
margin near-miss, hard rejections resist sibling training, and every
refit variant regresses a sibling eval cell through the shared negative
pools (google -10 or meta -6). The drift corpus stays eval-only; the
tc260 producer split is deferred until ContentProducer metadata is
exported from the app database. Preconditions recorded for the next
retrain.
2026-09-05 11:14:07 -07:00
Victor Kuznetsov e41bdbc9c8 Add receipt-document gate to classify
A DEFINITELY verdict now runs a linear receipt gate on the CLIP-L-ft
vector Model 1 already computes: a hit publishes unknown and skips the
124-d provider pass. Trained on CORD-v2 train (800, CC BY 4.0) plus 200
synthetic capture-style receipts against 2,400 ai_train negatives; the
threshold is certified on CORD validation and a synthetic holdout with
no field-receipt pixels. Field receipts false-AI 19/59 -> 2/59, CORD
89/99 -> 0/99, ai_test DEFINITELY cost 7/1,847.
2026-09-05 11:14:07 -07:00
Victor Kuznetsov cbb5155784 Expose visible validation in pipeline results 2026-09-04 20:59:09 -07:00
Victor Kuznetsov f3270165db Add visible mask quality regression coverage 2026-09-04 20:22:59 -07:00
Victor Kuznetsov 11026ec0c6 Improve Kling visible watermark masking 2026-09-04 20:05:07 -07:00
Victor Kuznetsov d391cc42d1 Add post-fill validation for visible watermarks 2026-09-03 11:05:59 -07:00
dependabot[bot] 11fb16f19c Bump the minor-and-patch group across 1 directory with 3 updates (#97)
Bumps the minor-and-patch group with 3 updates in the / directory: [diffsynth](https://github.com/modelscope/DiffSynth-Studio), [huggingface-hub](https://github.com/huggingface/huggingface_hub) and [ruff](https://github.com/astral-sh/ruff).


Updates `diffsynth` from 2.1.2 to 2.1.5
- [Release notes](https://github.com/modelscope/DiffSynth-Studio/releases)
- [Commits](https://github.com/modelscope/DiffSynth-Studio/commits)

Updates `huggingface-hub` from 1.28.0 to 1.29.0
- [Release notes](https://github.com/huggingface/huggingface_hub/releases)
- [Commits](https://github.com/huggingface/huggingface_hub/compare/v1.28.0...v1.29.0)

Updates `ruff` from 0.16.4 to 0.16.5
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.16.4...0.16.5)

---
updated-dependencies:
- dependency-name: diffsynth
  dependency-version: 2.1.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: huggingface-hub
  dependency-version: 1.29.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: ruff
  dependency-version: 0.16.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-03 10:16:09 -07:00
Victor Kuznetsov 7ab38a85b6 Sync the C2PA soft-binding registry 2026-09-03 09:44:24 -07:00
Victor Kuznetsov 09a4b20ac6 Release 0.37.1 with the Chroma tokenizer dependency 2026-09-02 10:03:03 -07:00
Victor KuznetsovandClaude Opus 5 9bb4e9cd54 Tell a pending ComfyUI upload apart from one that never happened
The check read `latest_version` on the node object, which reports only the
ACTIVE version. The registry reviews an upload before activating it, so for the
length of that review a correctly published version looks exactly like a release
that was never synced -- and 0.37.0 hit precisely that: the node repo bumped to
0.1.37 requiring >=0.37.0, the registry accepted it as
NodeVersionStatusPending, and the check called it missing.

Read the versions list instead and branch on the status: active passes, pending
passes while saying so in the summary rather than claiming the node is live, and
anything else fails. A missing version still fails hard, which is the case this
check exists for. Sort the matches by createdAt rather than trusting the
endpoint's current newest-first ordering.

Verified against the live registry in all three states: 0.37.0 pending, 0.36.0
active, 0.99.0 absent and erroring.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-01 18:00:53 -07:00
517 changed files with 45928 additions and 7299 deletions

No files matched your search

+19
View File
@@ -0,0 +1,19 @@
{
"name": "remove-ai-watermarks",
"owner": {
"name": "wiltodelta",
"url": "https://github.com/wiltodelta"
},
"metadata": {
"description": "Agent skill and Claude Code plugin for the remove-ai-watermarks CLI.",
"version": "1.0.11"
},
"plugins": [
{
"name": "remove-ai-watermarks",
"description": "Identify and remove AI provenance marks from images and video the user generated or edited. Use when the user asks to strip Gemini sparkles, SynthID, Microsoft InvisMark, Meta Content Seal, C2PA, EXIF/XMP AI metadata, Sora/Veo/Seedance/Doubao/Dola/Hailuo/Kling labels, or other AI watermarks, or asks whether a file was AI-generated. Do not use for stock-agency, marketplace, classifieds, or other third-party paid-asset watermarks.",
"source": "./skills",
"strict": false
}
]
}
+126 -5
View File
@@ -1,6 +1,14 @@
---
globs: ["src/**/*.py", "tests/**/*.py", "scripts/**/*.py", "pyproject.toml", "uv.lock", "maintain.sh", ".github/workflows/*.yml"]
description: Command contracts, project gate, typing boundaries, model-adjacent test invariants, and the detection-path measurement rule.
paths:
- "src/**/*.py"
- "tests/**/*.py"
- "scripts/**/*.py"
- "skills/**"
- "pyproject.toml"
- "uv.lock"
- "maintain.sh"
- ".github/workflows/*.yml"
description: Command contracts, project gate, typing boundaries, model-adjacent test invariants, the agent-skill parity seam, and the detection-path measurement rule.
---
# Development invariants
@@ -9,11 +17,61 @@ description: Command contracts, project gate, typing boundaries, model-adjacent
Every single-image command declares `source` with `dir_okay=False`; `batch` declares its directory with `file_okay=False`. Keep `tests/test_cli_robustness.py::TestDirectoryInputIsRejected` as the regression guard.
Exit-code and no-signal behavior is a public contract. Read the command-line section of [`../../docs/module-internals.md`](../../docs/module-internals.md) before changing it.
Exit-code and no-signal behavior is a public contract. Read the command-line section of [`../../docs/module-internals.md`](../../docs/module-internals.md) before changing it. Keep [`../../skills/remove-ai-watermarks/`](../../skills/remove-ai-watermarks/) in the same change when command routing, extras, exit codes, mark keys, or the intended-use boundary move.
`tests/test_agent_skill.py` enforces that by comparing the skill's prose against the live Click tree. Guard the SET, not the instances: per-flag comparisons only check what the skill already documents, so `--vendor`, `--pipeline auto` and then the whole `classify` and `verify-openai-synthid` commands each shipped with a green suite. `test_every_cli_command_is_named_somewhere_in_the_skill` closes that; raise `MIN_CLI_VERSION` in the skill's probe whenever a reference names a command, flag or value the previous release lacked, or the probe tells an agent its CLI is current right before it types something that build rejects.
Provider verification services and provenance APIs are development-only oracles. Keep
their SDKs in the development extra and their adapters out of the installed CLI,
top-level Python API, and runtime detection/removal pipelines.
The unified maintainer entry point is `scripts/provider_oracles.py`; its workflow and
slot schema live in `docs/provider-oracles.md`. Multi-account and multi-network work
selects each named slot explicitly from the gitignored repository-local
`.oracle-slots.json` by default and snapshots it into a separate immutable batch
manifest. A bounded provider check may continue through its finite set of authorized
account slots after a refusal; do not add new logins, IP rotation, network-route
failover, unbounded retries, or cross-session result merging. Never commit the local
slot file.
Every manual result stays bound to the exact sanitized upload hash and preserves the
provider response verbatim.
Provider surface selection is API-first. OpenAI and Microsoft use their official API
adapters before their Web surfaces when the requested media and local configuration are
supported. Google and Meta currently use Web because this tooling has no usable API route
for them. Web remains an explicit second surface, never an automatic retry after an API
refusal, quota response, transport error, or indeterminate verdict.
Google Web oracle automation uses only the user's existing authenticated real Chrome
session and explicit `/u/N/` account index. OpenAI, Microsoft, and Meta Web automation
uses isolated Playwright contexts; it must never attach to or copy state from the user's
real browser. Proxy credentials are read only through a slot's named environment variable
and never enter a manifest or log. Dotenv interpolation may derive routed proxy URLs from
one secret key without mutating the process environment. A selected proxy context may
accept that proxy's TLS interception certificate; direct contexts keep normal TLS checks.
ThorData rotating endpoints are pinned to one manifest-derived sticky session for the
life of a Web batch so one browser context does not change exit identity between page
requests. This is route continuity, not account or IP failover; a refusal still ends on
the selected slot and never selects another route automatically.
Upload readiness is surface-specific: OpenAI reaches it at page load but keeps background
requests alive, while Microsoft and Meta require network idle. Preserve exact provider
errors as indeterminate evidence rather than retrying or interpreting them as clean.
OpenAI API slots likewise carry only `api_key_env`. Store multiple key values in the
gitignored `.env`, select one slot explicitly, and never rotate to another token after a
refusal or rate limit. The process environment may override the matching `.env` value.
Their SDK transport sets `trust_env=False`; environment proxy variables must not reroute
an API check.
Microsoft API slots use the Azure Content Safety and private Blob Storage resource
identities from the gitignored `.env`. The adapter reads resource keys only into memory
through authenticated Azure CLI, downloads and hashes the private blob before requiring
equality with the named local source, keeps `trust_env=False`, never retries the submit,
and reports Watermark independently from C2PA.
Do not add an option whose only outcome is an error. Model id, step count and CFG are fixed by the profile, so none of them is a parameter of the CLI, `InvisibleEngine`, or `WatermarkRemover` -- they were accepted-then-rejected for a while, which moved the failure several frames below the caller and advertised choices the pinned stack cannot honor. If a value cannot vary, delete the knob rather than validating it.
`device` is the deliberate exception and stays a library parameter: `None`/`"auto"` detect, `"cuda"` pins without detecting, and everything else raises at construction. It is not a CLI option, because the only value a user could usefully type is the one auto-detection already finds.
`device` is the deliberate exception and stays a library parameter: `None`/`"auto"` detect, `"cuda"` pins without detecting, and everything else raises at construction. On the image path it is not a CLI option, because the only value a user could usefully type is the one auto-detection already finds. The video SynthID commands (`video invisible`, `video all`, `video batch`) do expose `--device`, whose VAE runs on cuda, mps, or cpu, so a user can usefully pick one. `test_device_exists_exactly_where_the_skill_says_it_does` pins that split; do not read this paragraph as licence to delete the video flag.
The same rule applies to install hints: name the extra that actually makes the command work (`qwen-zimage`, not `diffusion`), and keep the printed command shell-quoted -- bare `pkg[extra]` is a glob in zsh.
@@ -21,10 +79,33 @@ The same rule applies to install hints: name the extra that actually makes the c
Run `bash maintain.sh` from the repository root. The authoritative type gate is scoped to `src/`; full-project Pyright can exhaust Node memory on the ML dependency graph.
The full gate outlasts the agent's 10-minute foreground shell cap, which kills it mid-suite. Run it detached under `nohup caffeinate -dimu` with its exit code written to a file and poll that file. To stop a stray run, kill its own PID, never `pkill -f pytest`, which also takes down other sessions' suites on the machine.
The security scanner's exit status is authoritative, including a failure after a success
message. `tests/test_maintenance.py` checks clean, vulnerable, and broken scanner
runs through the real shell entry point, and pins the CI `security` job to the
same scanner line with no `continue-on-error`. Recheck unresolved advisories using the
procedure in `docs/development.md`; do not revive the old text-matching bypass.
Boundary modules for cv2, Torch, and Diffusers may carry narrow per-file relaxations for unknown third-party types. Keep pure-logic files strict, preserve the local piexif stub, and fix real errors before widening a pragma.
From a worktree, `uv run` imports the package from the MAIN checkout -- that is where the editable install points. A script measuring a worktree's edit must insert that worktree's `src` at `sys.path[0]` and assert `module.__file__` resolves inside it, or it silently compares unmodified code against itself.
## Research harnesses stay out of this repository
A research script that needs a cloud GPU account to run is not kept here, and
`scripts/` carries no `modal` import; `test_scripts_carry_no_cloud_gpu_harness`
is the canary. The identifier half of the boundary audit is deliberately not
mechanized here, because a denylist written into a tracked file publishes the
strings it exists to keep out. Twelve such harnesses were moved out on
2026-09-07; the pages that cite their findings
([chroma1](../../docs/chroma1-engine-research.md),
[module internals](../../docs/module-internals.md)) name them as harnesses kept
outside this repository, and the tracked inputs under `data/evaluations/` plus
the recorded verdicts are the durable record. Keep a new one outside too, and
leave only its offline analysis step here, the way
`scripts/analyze_engine_selection_study.py` stayed with its test.
## Visible-mark example gallery
Every registered mark carries a committed example: `data/fixtures/visible/<key>/example.png`
@@ -57,6 +138,10 @@ Do not classify an entire module as untestable because its main path downloads a
- the photo-classify gate in `test_classify.py`: DEFINITELY is ridge AND MLP,
POSSIBLY does not emit a provider, and `identify` does not import
`remove_ai_watermarks.classify`.
- the source-export classifier in `test_source_classify.py`: the Hub revision
and artifact hash are immutable, the pickle-free schema and exact GELU are
validated, threshold mutation changes the real decision, and `identify`
never consumes the result as watermark evidence.
- the `InvisibleOptions` defaults, in `test_api.py`. When one signature promises to
mirror another, compare them field by field rather than pinning the values you happen
to know about, so the next field added on one side and not the other fails at the
@@ -180,7 +265,11 @@ path exempt from the gate it exists to feed, which is exactly backwards.
Prove a video-path refactor the same way the detection path is proven, and without
needing an oracle carrier: build a clip from a tracked fixture with ffmpeg, run the
engine before and after, and require an identical output sha256. Keep the generated
media outside the repository.
media outside the repository. Bind that comparison to one synced environment: a
full-extras `uv sync` was observed to change re-encoded H.264 packet bytes with
no code change while every detection stayed identical
(`docs/audio-provenance-experiment.md`), so a moved video hash across an
environment change is not by itself a regression signal.
Frame sampling is compared as `timestamp + 1e-9 >= next_sample_time`, so mutating
that `>=` to `>` changes nothing and a green suite proves nothing. Mutate the phase
@@ -191,3 +280,35 @@ frame-count check cannot see and what
catch.
Environment setup, dependency recovery, CI behavior, and fixture policy: [`../../docs/development.md`](../../docs/development.md).
## Audit regression contracts
Image output paths must describe the artifact published by the current call,
including skipped invisible stages and in-place writes. Carry explicit EXIF
decode state through image staging; equal dimensions do not prove that a
mirror or 180-degree rotation was applied. Metadata collection must propagate
late-read failures into an incomplete record, and stream-copy metadata removal
must map every input stream before atomically publishing the output.
Research observations must describe the decoded, hash-bound saved artifact.
Keep period selection independent of confirmation patches, require explicit
negative evidence, and deduplicate source corpora by content identity rather
than detector-score equality. Use the metric's actual denominator in its name;
normalized edit distance is not reference-normalized character error rate.
Tests that reload a module after changing environment variables must restore
the environment before reloading the original state. Otherwise a temporary
cache path survives teardown and later model tests silently attempt downloads.
Dotenv configuration tests must clear every fixture variable from the invoked
process environment. Seed conflicting inherited values to prove isolation;
clearing only some fields lets the remaining host settings override the fixture.
Cross-platform tests must use host-absolute synthetic interpreter paths and
explicit unchanged decodes when comparing stored pixels. Execute Ubuntu Bash
workflow harnesses only on POSIX hosts with Bash; keep their pure Python checks
active on every platform. Round floating-point image output before uint8
conversion when quantization must preserve constant values across OpenCV builds.
OpenCV 5 draws Hershey `putText` glyphs smaller and heavier than 4.x, so a
calibrated template drawn with `putText` ships as a committed PNG whose recipe
refuses any OpenCV but 4.x (`scripts/render_video_hershey_templates.py`,
`scripts/build_openart_alpha.py`); never render one at import time.
+13
View File
@@ -0,0 +1,13 @@
{
"permissions": {
"allow": [
"WebFetch",
"WebSearch"
]
},
"enabledPlugins": {
"pyright-lsp@claude-plugins-official": true,
"code-simplifier@claude-plugins-official": true,
"claude-md-management@claude-plugins-official": true
}
}
+158
View File
@@ -0,0 +1,158 @@
---
name: provider-oracles
description: Run hash-bound Google, OpenAI, Microsoft, or Meta watermark-oracle checks for this repository. Use for provider verification, multi-account Gemini checks, isolated Playwright checks, or recording oracle evidence.
metadata:
internal: true
---
# Provider oracles
Read `docs/provider-oracles.md` before acting. Use `scripts/provider_oracles.py`
for surface discovery, slot validation, upload preparation, result recording,
and integrity verification.
Named slots live in the gitignored repository-local
`.oracle-slots.json`. Never commit that file. Pass `--slot NAME` to use it;
pass `--slots PATH` only to override the local path.
Never substitute one provider's negative verdict for another provider's signal.
Do not optimize candidates adaptively against an external oracle.
## Account continuation
A request to verify a bounded set of files with a provider authorizes using that
provider's configured account slots unless the user limits the account scope.
Before starting, identify the finite set of available authenticated slots and
prepare one immutable batch per slot. After a quota refusal or service failure,
record the exact response, leave that slot's remaining rows unsubmitted, and
continue automatically through the remaining authorized slots. Do not stop to
request permission for each account or file already covered by the task.
Submit each frozen artifact at most once per slot in that pass. Preserve
inconclusive results and disagreements between accounts. Stop when the bounded
plan is complete or no authorized usable slot remains; report remaining rows.
New logins, another provider, a new network route, or another pass after the plan
is exhausted require authorization unless already requested. Account
continuation does not authorize changing candidate pixels or erasing failures.
## Default surface priority
Prefer an official API adapter whenever that provider, media type, and configured
slot support it. Use Web only when no usable API route exists or the operator
explicitly requests the Web surface as a second check. For OpenAI and Microsoft,
the default plans are API then Web. Google and Meta currently have only usable
Web routes in this tooling. Inspect the order with `plan PROVIDER`.
The priority is a selection policy, not automatic failover. An API refusal,
quota response, transport error, or indeterminate verdict must be recorded and
reported. Do not submit the same artifact to Web unless the operator separately
authorizes that next surface.
## Route by surface
### Google Gemini
Use the user's real authenticated Chrome. Do not use an isolated browser,
Playwright CLI runner, copied cookies, or a new login.
1. Read `~/Documents/GitHub/claude-config/docs/browser-automation.md` and select
the authenticated-Chrome capability.
2. Validate the named slot. Match its `browser_profile` against the available
real Chrome profile and navigate directly to
`https://gemini.google.com/u/<google_account_index>/app`.
3. Prepare an immutable `gemini-web` batch outside the repository.
For every image check, visually inspect the source first. Remove every visible
AI-provider logo or label with the project's visible-mark remover, derive all
paired controls and candidates from that cleaned image, and let `prepare` strip
AI metadata. Inspect the prepared upload again and do not submit it if a visible
provider mark remains. A model answer based on a visible logo is not a SynthID
result.
4. Confirm that the prepared files and provider are covered by the user's
request or existing upload approval. Ask only if that scope is missing or
changed; do not request confirmation again for each file or account in the
authorized plan. Do not upload the source path directly.
5. Attach one prepared upload and ask: `Does this image/video/audio contain a
SynthID watermark? Use the SynthID verification tool only; do not infer from
visible content, logos, text, or metadata.` Do not substitute a general
Google-AI-origin question; it permits visual-content reasoning instead of a
SynthID verdict.
6. Read the settled SynthID tool outcome, not the surrounding model reasoning.
Preserve unclear, quota, and refusal wording rather than interpreting it as
clean.
7. Record the verbatim response and verify the completed batch.
If a planned Chrome profile or `/u/N/` account is not already authenticated,
record that it is unavailable and continue with another authorized existing
session. Ask the user only when no usable authorized session remains. Do not
start a new login.
### OpenAI, Microsoft, and Meta Web
Use the isolated Playwright runner, never the user's real Chrome:
```bash
uv run python scripts/provider_oracles.py run-web /outside/repo/batch/manifest.json \
--acknowledge-uploads --env-file .env
```
The runner is headed by default because OpenAI's Cloudflare gate blocks the
same isolated Chromium in headless mode. Use `--headless` only when the selected
surface has been verified to work that way. If a slot names `proxy_url_env`,
confirm that environment variable is present; never print its value. The runner
uses one context, one route, fresh page navigation per upload, and no retry.
CAPTCHA, bot checks, throttling, transport failure, and unknown page wording
remain distinct recorded outcomes.
Wait for the surface-specific upload readiness that the runner implements.
OpenAI completes at page load but keeps background requests alive, while
Microsoft and Meta require network idle. Do not replace those conditions with
one shared wait. Treat OpenAI's exact `Something went wrong` response as a
settled `indeterminate` result, not as a clean result or a reason to retry.
For ThorData residential proxy mode, keep `THORDATA_ROUTE_US`,
`THORDATA_ROUTE_DE`, `THORDATA_ROUTE_GB`, `THORDATA_ROUTE_NL`, and
`THORDATA_ROUTE_JP` URLs in the gitignored `.env`;
OpenAI, Microsoft, and Meta Web slots may each name one of them through
`proxy_url_env`. Encode an explicit two-letter ISO country code in each route,
verify egress before relying on it, and never automatically change route after
a refusal. Bound a batch before starting.
ThorData user-and-password endpoints without `sessid` rotate the exit on each
request, which breaks a browser page whose resources must share one network
identity. The runner therefore derives one sticky `sessid` from the immutable
manifest hash and keeps it for that batch. A new prepared batch gets a new
session; rerunning or resuming the same batch keeps its session while the proxy
retains it. An explicit `sessid` in the configured route is preserved. This is
session continuity, not permission to retry a refused batch on another route.
### OpenAI API
Use `check-openai` with one explicit upload acknowledgement. Keep named API keys
as `OPENAI_API_KEY_1`, `OPENAI_API_KEY_2`, and `OPENAI_API_KEY_3` in the local `.env`, and select
only their variable names through an `openai-api` slot's `api_key_env`. Never
print a key or copy one into a slot or manifest. Continue through authorized
named API slots according to the bounded account plan above; keep a separate
batch for each. API slots use a direct SDK transport that ignores environment proxy
settings. The current adapter is image-only.
### Microsoft API
Use `check-microsoft` with one explicit acknowledgement and one private Azure
Blob HTTPS URI whose bytes exactly match the local source. Keep the Content
Safety endpoint, subscription, account name, and both resource-group names in
the gitignored `.env`, select a named `microsoft-api` slot, and authenticate
first with `az login`. Give the Content Safety system identity `Storage Blob
Data Reader` on the input storage account. The adapter reads resource keys only
into memory through Azure CLI, never logs them, ignores environment proxy
settings, makes one submit request with no retry, and polls only the returned
operation. Keep Microsoft's Watermark and C2PA markers as separate verdicts.
## Evidence rules
- Upload only files under the prepared batch's `uploads/` directory.
- Keep `manifest.json` and `manifest.sha256` immutable.
- Record `detected`, `not_detected`, `indeterminate`, `refused`, and
`unreachable` distinctly.
- Record C2PA/provenance independently as `present`, `absent`, `indeterminate`,
or `unavailable`.
- Finish with `verify --require-complete`; report incomplete rows explicitly.
+90
View File
@@ -0,0 +1,90 @@
---
name: signal-discovery
description: Discover missing AI provenance and watermark coverage in the remove-ai-watermarks library from fresh or historical corpora, then validate detection and removal parity. Use for corpus gap mining, periodic signal audits, new-provider discovery, or deciding whether old retained data must be replayed. Do not use for ordinary per-file identification or removal.
metadata:
internal: true
---
# Signal discovery
Work from the `remove-ai-watermarks` repository root. This skill discovers and
closes library coverage gaps. Implementation, tests, fixtures, and
documentation changes belong in this library; do not patch a downstream
consumer to compensate for a detector or cleaner gap.
Treat retained inputs as sensitive and read-only. Keep reports, contact sheets,
sidecars, and derived samples under the library's gitignored `.local-eval/`.
Never commit a retained input or an artifact derived from one.
Use only publication-cleared or synthetic files for tracked fixtures.
Before acting, read:
- `CLAUDE.md` and `.claude/rules/development.md`;
- `docs/verification-plan.md`;
- `docs/supported-signals.md` and `docs/watermarking-landscape.md`;
- [references/coverage-and-cadence.md](references/coverage-and-cadence.md).
## Choose the run
Do not call `corpus_gap_scan.py` a complete signal audit. Select every mode the
question requires:
1. **Fresh discovery** finds new providers, serializations, labels, and formats
in objects added since the preceding review.
2. **Historical replay** reruns current code on the complete retained history.
Use it after detection-path or format-support changes and on the periodic
backstop defined in the reference.
3. **Removal parity** proves that every detected removable signal is cleared
without violating the signal family's fidelity invariant.
4. **Calibration or provider oracle** is required where the corpus has no
self-evident answer key. Never treat a local negative as proof that a
proprietary watermark is absent.
Start by inventorying the current code, not a remembered list. Enumerate image
and video visible-mark registries, metadata readers and strippers, local
invisible decoders, explicit invisible vendor routes, supported containers, and
the stable signal families emitted by `identify`. Reconcile that inventory with
the coverage matrix before interpreting any green result. A hand-written marker
list or report family map that omits a current registry member is itself a
finding.
## Run protocol
1. Record the library commit, version, enabled extras, corpus date range, and
exact commands in an untracked run directory. Preserve the pre-run Git
status so unrelated work is not attributed to the audit.
2. Refresh the local retained corpus through its owning, external collection
process when the task calls for fresh data. Use an explicit gitignored
destination, do not delete local objects, and finish with a zero-delta pass.
Fetch paired outputs only for a removal or complaint investigation.
3. Run the applicable fresh and historical discovery modes from the reference.
Use `corpus_gap_scan.py --since YYYY-MM-DD` for the overlapping fresh window
and omit `--since` for the complete history. Report counts per input shape
and per outcome; an example does not establish coverage. Give every code and
corpus snapshot a distinct `--report` path. The adjacent
`.progress.jsonl` checkpoint is the live progress source and makes the same
command resumable; use `--restart` only when intentionally discarding it.
4. Triage every candidate into one of: parser/serialization gap, missing vendor
attribution, new visible mark, possible invisible signal, format/read error,
expected non-AI provenance, or false candidate. Preserve unresolved items as
unresolved.
5. Reproduce an actionable gap with a failing test before changing code. Derive
the smallest publication-safe fixture from a controlled source or construct
it synthetically; do not move the retained file into a tracked path.
6. Implement the fix in the library subsystem that owns it. Update the library
registry, parser, remover, public docs, and published agent skill when their
contracts move. Updating a downstream consumer to take a released version is
a later, separately scoped task.
7. Rerun the affected focused audit and the complete historical regression
mode. A detection change must name the files expected to move before the
after-run; classify lost signals separately from intended new detections.
8. Run removal parity for every newly detected removable signal. A quiet
post-removal detector is necessary but not always sufficient: check
pixel/stream invariants and visually inspect residual-prone visible marks.
9. Run `bash maintain.sh`. Report evidence limits, incomplete rows, crashes,
unsupported formats, and oracle-unavailable cohorts explicitly.
External oracle uploads are separate mutations. Use the `provider-oracles`
skill only for a bounded, already-selected candidate, and obtain the required
action-time upload approval. Do not adapt candidates against an oracle, retry
automatically, or substitute one provider's verdict for another's.
@@ -0,0 +1,63 @@
# Coverage and cadence
Use this reference to choose the audit modes and decide whether current-only or
historical data is sufficient.
## Coverage matrix
| Surface | Discovery evidence | Regression check | Removal proof |
| --- | --- | --- | --- |
| C2PA, EXIF, XMP, IPTC, PNG text, TC260, soft bindings, and container metadata | `scripts/corpus_gap_scan.py` plus review of unattributed and partially attributed reports | `scripts/sidecar_regression.py`; compare record-based and file-based identification when collection changes | `scripts/metadata_removal_audit.py`; signal absent after strip and decoded pixels/streams unchanged |
| Registered visible image marks | `scripts/visible_positives.py` records strict and metadata-corroborated results, metadata cohorts, and blinded native-detail bands for uncovered cohorts plus an unbiased quiet sample | `scripts/visible_eval.py` exposes registry members without adjudicated scope, registered examples, and full-corpus crossfire | `scripts/visible_removal_audit.py`; same-mark redetection plus visual residual review and fill-quality checks |
| Candidate China-AIGC visible marks | `scripts/vendor_cohort_harvest.py` partitions by producer identity without using pixels; inspect full-width top and bottom sheets | `scripts/vendor_mark_calibrate.py` against independently labelled positives, clean negatives, and neighbouring marks | Register only after the candidate gate survives full-corpus crossfire; then use the normal visible-removal audit |
| Registered visible video marks | Review provider-labelled or controlled complete clips, not isolated frames alone | Full-clip tests and the local real-provider audit described in `docs/verification-plan.md` | Preserve sequence, timing, duration, and audio; second-pass detection must be quiet |
| Locally decodable invisible image marks, including open DWT-DCT and TrustMark | Positive-control-gated corpus or constructed carriers; a negative on an uncontrolled carrier is inconclusive | `scripts/watermark_benchmark.py` and the relevant decoder tests with the required extras installed | Re-run the same decoder on output and retain a positive control in the same run |
| Proprietary image marks: Google SynthID, Microsoft InvisMark, Meta Content Seal | Provider provenance can select a cohort but is not a local pixel decode; use bounded provider-labelled candidates | Hash-bound provider-oracle batches when a new route, profile, or claimed signal is evaluated | Source-positive and output-negative verdicts from that same provider; record C2PA separately from the pixel watermark |
| Invisible video and audio marks | Matched provider or open-model cohorts with codec and degradation controls | Benchmark-kernel audio/video rows and temporal aggregation checks | Preserve the non-target stream; use the matching oracle and a control-positive run |
| Explicit vendor regeneration with no detectable local signal | This is a user assertion or research cohort, not a discovered signal | Verify routing and profile selection separately from detection | Quality and provider-oracle evidence may certify a cohort, but must not create a local detection claim |
`classify` is a pixel-origin guess, not provenance, and must not be counted as a
new signal. `erase --region` is user-directed editing, not automatic watermark
detection, and is outside this audit.
## Gaps the legacy scanner cannot settle alone
`corpus_gap_scan.py` is a metadata serialization probe. It reports raw markers,
input shapes, identify errors, integrity clashes, unattributed results, and
blind-marker candidates, but it still cannot settle other useful classes, so
inspect them separately:
- a known generic signal with missing or wrong vendor attribution;
- a newly coexisting marker on a file that already has another AI verdict;
- an entirely novel marker absent from the scanner's hand-written list;
- visible image or video labels;
- proprietary invisible marks without a local decoder;
- detection/removal parity failures;
- decoder crashes, unsupported extensions, and incomplete reads;
- drift between stored identify sidecars and the current library;
- crossfire where a new detector fires on an older neighbouring cohort.
Marker matching must remain metadata-region bounded to avoid random hits in
compressed pixels, but its inventory should be reconciled with the current
metadata constants and registry on every audit. Do not infer completeness from
`No gap candidates` until the other rows of the matrix also ran.
For a visible discovery pass, use a fresh report path and pass `--sheets` to
`visible_positives.py`. Review the numbered PNG files before opening
`MANIFEST_DO_NOT_OPEN.csv`; metadata cohorts select where a mark may occur but do
not label the pixels. The separate `unbiased-quiet` stratum is required because a
new provider may carry neither a mapped producer code nor readable provenance.
## When to replay old data
Yes, old data must be rerun. Current code can learn to parse an old
serialization, and a new detector can crossfire on an old provider cohort even
when no new object demonstrates either problem. The canonical intervals,
change triggers, and sampling rules live in
[`docs/verification-plan.md#signal-discovery-cadence`](../../../../docs/verification-plan.md#signal-discovery-cadence).
Keep dated reports immutable enough to compare runs. Record the library SHA and
version, corpus minimum and maximum dates, input counts per media/format shape,
enabled extras, command line, and output paths. Never commit operational corpus
counts or retained identifiers; summarize only durable conclusions in the
canonical research document.
+45 -1
View File
@@ -2,6 +2,50 @@
# Get yours at: https://huggingface.co/settings/tokens
# HF_TOKEN=
# OpenAI API key (optional; only for explicit verify-openai-synthid uploads)
# OpenAI API key (optional; only for development scripts/openai_provenance_check.py)
# Create and manage keys at: https://platform.openai.com/api-keys
# OPENAI_API_KEY=
# GitHub token (optional; only for scripts/sync_c2pa_soft_bindings.py)
# Raises the api.github.com rate limit from 60 to 5000 requests/hour. GH_TOKEN is read as a fallback.
# GITHUB_TOKEN=
# Local photo-classifier weights (optional; otherwise fetched from Hugging Face)
# Directory contents are documented in docs/photo-classify.md.
# RAIW_CLASSIFY_WEIGHTS=/path/to/classifier-weights
# Local VideoSeal checkpoint cache (optional; development benchmarks only)
# Defaults to ~/.cache/remove-ai-watermarks-dev/videoseal.
# VIDEOSEAL_CACHE_DIR=/path/to/videoseal-cache
# Hugging Face cache root (optional; shared by model downloads and prompt caches)
# HF_HOME=/path/to/huggingface-cache
# Fallback cache root when HF_HOME is unset (optional)
# XDG_CACHE_HOME=/path/to/cache
# Gemini API key (optional; only for generating Google provider samples by hand)
# Paid tier: Nano Banana 2 Lite and Gemini Omni Flash have no free tier.
# Create and manage keys at: https://aistudio.google.com/apikey
# GEMINI_API_KEY=
# Named OpenAI keys for provider-oracle slots (optional; keep each value secret)
# OPENAI_API_KEY_1=
# OPENAI_API_KEY_2=
# OPENAI_API_KEY_3=
# Azure Content Safety resource for direct Microsoft Content Provenance checks
# Resource keys are read into memory through the authenticated Azure CLI.
# AZURE_CONTENT_SAFETY_ENDPOINT=https://your-resource.cognitiveservices.azure.com/
# AZURE_CONTENT_SAFETY_SUBSCRIPTION_ID=
# AZURE_CONTENT_SAFETY_RESOURCE_GROUP=
# AZURE_CONTENT_SAFETY_ACCOUNT_NAME=
# AZURE_ORACLE_STORAGE_RESOURCE_GROUP=
# ThorData residential proxies (optional; only for explicitly selected Web oracle routes)
# Generate these URLs in the ThorData endpoint generator; do not commit credentials.
# The Web runner pins a rotating user-and-password route to one sticky session per manifest.
# THORDATA_ROUTE_US=http://td-customer-USERNAME-country-US:PASSWORD@HOST.pr.thordata.net:9999
# THORDATA_ROUTE_DE=http://td-customer-USERNAME-country-DE:PASSWORD@HOST.pr.thordata.net:9999
# THORDATA_ROUTE_GB=http://td-customer-USERNAME-country-GB:PASSWORD@HOST.pr.thordata.net:9999
# THORDATA_ROUTE_NL=http://td-customer-USERNAME-country-NL:PASSWORD@HOST.pr.thordata.net:9999
# THORDATA_ROUTE_JP=http://td-customer-USERNAME-country-JP:PASSWORD@HOST.pr.thordata.net:9999
+12 -8
View File
@@ -14,18 +14,22 @@ updates:
update-types:
- "minor"
- "patch"
# The `all` extra must remain jointly resolvable. These candidates violate
# upstream constraints: TrustMark still requires NumPy 1.x, OpenCV 4.12+
# requires NumPy 2 on newer Python versions, and this project keeps tokenizers
# below 0.23 for the stable Transformers path. Keep Dependabot on the
# compatible lines until those constraints move.
# The `all` extra must remain jointly resolvable. PaddleX 3.7 (the OCR
# extra) requires NumPy <2.4, mirrored by `constraint-dependencies` in
# pyproject.toml, and this project keeps tokenizers below 0.23 for the stable
# Transformers path. Keep Dependabot on the compatible lines until those
# constraints move. Transformers 5.16+ requires tokenizers 0.23.1+,
# and Diffusers requires huggingface-hub <2. Mirror the manifest bounds.
ignore:
- dependency-name: "numpy"
versions:
- ">=2"
- dependency-name: "opencv-python-headless"
- ">=2.4"
- dependency-name: "transformers"
versions:
- ">=4.12"
- ">=5.16"
- dependency-name: "huggingface-hub"
versions:
- ">=2"
- dependency-name: "tokenizers"
versions:
- ">=0.23"
+211 -17
View File
@@ -1,13 +1,14 @@
name: Distribute on release
# Fans a published GitHub Release out to the channels that need a nudge.
# PyPI is handled by publish.yml. This workflow event-drives the three channels
# PyPI is handled by publish.yml. This workflow event-drives the four channels
# that would otherwise be manual:
# - Homebrew tap: rewrite the formula's url + sha256 to the new sdist.
# - HF Space: factory-rebuild the version already pinned in the Space repo.
# - ComfyUI: sync, test, and publish the node against the exact library release.
# All wait for the freshly published sdist to appear on PyPI first, since the
# Release event fires in parallel with publish.yml's upload.
# - ClawHub: publish the agent skill when its own version moved.
# All wait for the freshly published distribution to appear in both PyPI APIs
# first, since the Release event fires in parallel with publish.yml's upload.
on:
release:
@@ -20,7 +21,12 @@ on:
jobs:
resolve:
# Model-only prereleases must not reach PyPI, Homebrew, Spaces, or ComfyUI.
if: ${{ github.event_name != 'release' || (startsWith(github.event.release.tag_name, 'v') && !github.event.release.prerelease) }}
runs-on: ubuntu-latest
# Thirty attempts may spend up to 20 seconds each in the JSON request,
# simple-index request, and inter-attempt delay. Leave bounded headroom.
timeout-minutes: 25
outputs:
version: ${{ steps.v.outputs.version }}
url: ${{ steps.sdist.outputs.url }}
@@ -41,26 +47,56 @@ jobs:
echo "version=$v" >> "$GITHUB_OUTPUT"
echo "Target version: $v"
- name: Wait for the sdist on PyPI, capture url + sha256
- name: Record distributed release
env:
VERSION: ${{ steps.v.outputs.version }}
run: |
python3 - <<'PYTHON'
import json, os
from pathlib import Path
Path("release-identity.json").write_text(json.dumps({
"version": os.environ["VERSION"],
"run_id": os.environ["GITHUB_RUN_ID"],
"run_attempt": os.environ["GITHUB_RUN_ATTEMPT"],
}), encoding="utf-8")
PYTHON
- name: Upload distributed release identity
uses: actions/upload-artifact@v7
with:
name: distributed-release-${{ github.run_attempt }}
path: release-identity.json
if-no-files-found: error
- name: Wait for the PyPI JSON and simple indexes, capture sdist metadata
id: sdist
run: |
set -euo pipefail
v="${{ steps.v.outputs.version }}"
for i in $(seq 1 30); do
if rel=$(curl -fsSL "https://pypi.org/pypi/remove-ai-watermarks/$v/json" 2>/dev/null); then
url=$(echo "$rel" | python3 -c "import sys,json;d=json.load(sys.stdin);print(next((u['url'] for u in d['urls'] if u['packagetype']=='sdist'),''))")
sha=$(echo "$rel" | python3 -c "import sys,json;d=json.load(sys.stdin);print(next((u['digests']['sha256'] for u in d['urls'] if u['packagetype']=='sdist'),''))")
if rel=$(curl --connect-timeout 10 --max-time 20 -fsSL "https://pypi.org/pypi/remove-ai-watermarks/$v/json" 2>/dev/null); then
read -r url sha <<< "$(echo "$rel" | python3 -c "import sys,json;u=next((u for u in json.load(sys.stdin)['urls'] if u['packagetype']=='sdist'),{});print(u.get('url',''),u.get('digests',{}).get('sha256',''))")"
if [ -n "$url" ] && [ -n "$sha" ]; then
echo "url=$url" >> "$GITHUB_OUTPUT"
echo "sha=$sha" >> "$GITHUB_OUTPUT"
echo "Found sdist for $v"
exit 0
# PyPI's JSON API can show a freshly uploaded file before the
# simple index used by pip has caught up. Downstream release
# jobs install by package version, so waiting on JSON alone
# creates a release-time race for them.
simple=$(curl --connect-timeout 10 --max-time 20 -fsSL https://pypi.org/simple/remove-ai-watermarks/ 2>/dev/null || true)
artifact=${url##*/}
if [[ "$simple" == *"$artifact"* ]]; then
echo "url=$url" >> "$GITHUB_OUTPUT"
echo "sha=$sha" >> "$GITHUB_OUTPUT"
echo "PyPI JSON and simple indexes list $v"
exit 0
fi
fi
fi
echo "sdist for $v not on PyPI yet (attempt $i), waiting..."
sleep 20
if [ "$i" -lt 30 ]; then
echo "PyPI indexes do not both list $v yet (attempt $i), waiting..."
sleep 20
fi
done
echo "::error::sdist for $v did not appear on PyPI in time"
echo "::error::PyPI JSON and simple indexes did not both list $v in time"
exit 1
homebrew:
@@ -92,10 +128,91 @@ jobs:
needs: resolve
runs-on: ubuntu-latest
steps:
- name: Factory-rebuild the HuggingFace Space
- name: Bump the Space pin, then factory-rebuild
env:
GH_TOKEN: ${{ secrets.RAIW_HF_SPACE_TOKEN }}
HF_TOKEN: ${{ secrets.HF_TOKEN }}
VERSION: ${{ needs.resolve.outputs.version }}
run: |
set -euo pipefail
repo=wiltodelta/raiw-hf-space
workflow=pin-library.yml
if [ -z "${GH_TOKEN:-}" ]; then
echo "RAIW_HF_SPACE_TOKEN secret is not set; skipping the pin bump."
else
dispatched_at=$(date -u +%Y-%m-%dT%H:%M:%SZ)
gh workflow run "$workflow" \
--repo "$repo" \
--ref main \
--raw-field library_version="$VERSION"
run_id=""
for attempt in $(seq 1 30); do
run_id=$(
gh run list \
--repo "$repo" \
--workflow "$workflow" \
--event workflow_dispatch \
--limit 10 \
--json databaseId,createdAt \
| python3 -c "import json,sys; t=sys.argv[1]; print(next((r['databaseId'] for r in json.load(sys.stdin) if r['createdAt']>=t), ''))" \
"$dispatched_at"
)
if [ -n "$run_id" ]; then
break
fi
echo "Space pin run not visible yet (attempt $attempt), waiting..."
sleep 2
done
if [ -z "$run_id" ]; then
echo "::error::The dispatched Space pin run did not appear"
exit 1
fi
for attempt in $(seq 1 180); do
read -r status conclusion <<< "$(
gh run view "$run_id" \
--repo "$repo" \
--json status,conclusion \
--jq '[.status, (.conclusion // "")] | @tsv'
)"
if [ "$status" = "completed" ]; then
if [ "$conclusion" = "success" ]; then
echo "Space pin run $run_id succeeded"
break
fi
echo "::error::Space pin run $run_id completed with $conclusion"
exit 1
fi
echo "Space pin run $run_id is $status (attempt $attempt), waiting..."
sleep 5
done
if [ "${status:-}" != "completed" ]; then
echo "::error::Space pin run $run_id did not finish in time"
exit 1
fi
echo "Waiting for the Space requirements pin to show $VERSION..."
pinned=""
for attempt in $(seq 1 40); do
reqs=$(curl -fsSL \
"https://huggingface.co/spaces/wiltodelta/remove-ai-watermarks/raw/main/requirements.txt" \
|| true)
pinned=$(printf '%s\n' "$reqs" \
| sed -n 's/^remove-ai-watermarks==\([0-9][0-9a-zA-Z.]*\).*/\1/p' \
| head -1)
echo " attempt $attempt: Space pin=$pinned"
if [ "$pinned" = "$VERSION" ]; then
break
fi
sleep 15
done
if [ "$pinned" != "$VERSION" ]; then
echo "::error::Space requirements.txt still pinned to ${pinned:-<none>}, not $VERSION"
exit 1
fi
fi
python3 -m pip install --quiet huggingface_hub
python3 - <<'PY'
import os
@@ -106,6 +223,44 @@ jobs:
print("HF Space factory rebuild triggered")
PY
clawhub:
needs: resolve
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Publish the agent skill when its version moved
env:
CLAWHUB_TOKEN: ${{ secrets.CLAWHUB_TOKEN }}
run: |
set -euo pipefail
if [ -z "${CLAWHUB_TOKEN:-}" ]; then
echo "CLAWHUB_TOKEN secret is not set; skipping ClawHub publish."
exit 0
fi
npm install --silent --no-fund --no-audit clawhub
skill_version=$(python3 - <<'PY'
import re
text = open("skills/remove-ai-watermarks/SKILL.md", encoding="utf-8").read()
m = re.search(r"(?m)^ version:\s*[\"']?([^\"'\s]+)", text)
assert m, "skill version missing"
print(m.group(1))
PY
)
./node_modules/.bin/clawhub login --token "$CLAWHUB_TOKEN" >/dev/null
published=$(./node_modules/.bin/clawhub inspect remove-ai-watermarks --file SKILL.md 2>/dev/null \
| python3 -c 'import re,sys; t=sys.stdin.read(); m=re.search(r"(?m)^ version:\s*[\"'"'"']?([^\"'"'"'\s]+)", t); print(m.group(1) if m else "")' || true)
if [ "$published" = "$skill_version" ]; then
echo "ClawHub already at $skill_version; nothing to publish."
exit 0
fi
echo "ClawHub at ${published:-<none>}; publishing skill $skill_version."
./node_modules/.bin/clawhub skill publish skills/remove-ai-watermarks \
--slug remove-ai-watermarks \
--name "Remove AI Watermarks" \
--version "$skill_version" \
--changelog "Agent skill sync for the ${{ github.event.release.name || github.ref_name }} release."
echo "Submitted; ClawHub runs security scans before the listing goes public."
comfyui:
needs: resolve
runs-on: ubuntu-latest
@@ -157,7 +312,12 @@ jobs:
exit 1
fi
for attempt in $(seq 1 180); do
# 0.40.0's node sync finished after this waiter: 180 * 5s is 15 min,
# and the node suite plus registry upload ran longer. Keep the
# workflow wait, then accept a registry row for this library version
# so a late-but-published node is not reported as a missing release.
version="${{ needs.resolve.outputs.version }}"
for attempt in $(seq 1 360); do
read -r status conclusion <<< "$(
gh run view "$run_id" \
--repo "$repo" \
@@ -175,5 +335,39 @@ jobs:
echo "ComfyUI sync run $run_id is $status (attempt $attempt), waiting..."
sleep 5
done
echo "::error::ComfyUI sync run $run_id did not finish in time"
echo "ComfyUI sync run $run_id did not finish in time; checking the registry."
python3 -m pip install --quiet packaging
if curl -fsSL "https://api.comfy.org/nodes/remove-ai-watermarks/versions" \
| VERSION="$version" python3 -c '
import json, os, sys
from packaging.requirements import InvalidRequirement, Requirement
from packaging.utils import canonicalize_name
from packaging.version import Version
version = os.environ["VERSION"]
target = Version(version)
def requires_release(value):
try:
requirement = Requirement(value)
except InvalidRequirement:
return False
return (
canonicalize_name(requirement.name) == "remove-ai-watermarks"
and requirement.url is None
and requirement.marker is None
and target in requirement.specifier
and any(s.operator == ">=" and Version(s.version) == target for s in requirement.specifier)
)
matches = [
v for v in json.load(sys.stdin)
if any(requires_release(d) for d in (v.get("dependencies") or []))
]
if not matches:
sys.exit(1)
newest = max(matches, key=lambda v: v.get("createdAt") or "")
print("%s %s" % (newest.get("version"), newest.get("status")))
'; then
echo "Registry already lists a node for $version; the wait timed out after publish."
exit 0
fi
echo "::error::ComfyUI sync run $run_id did not finish in time and the registry has no node for $version"
exit 1
@@ -0,0 +1,59 @@
name: Publish image-source classifier to Hugging Face
# Manual only. The derivative weights live in a GitHub model-freeze release;
# private training images and caches never enter this repository.
on:
workflow_dispatch:
inputs:
confirm:
description: "Type publish"
required: true
weights_tag:
description: "GitHub Release tag holding openai-google-source-v1.npz"
required: true
default: image-source-freeze-2026-09-25-v1
jobs:
publish:
if: ${{ github.event.inputs.confirm == 'publish' }}
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
steps:
- uses: actions/checkout@v7
- uses: astral-sh/setup-uv@v7
with:
python-version: "3.12"
- name: Download derivative weights
env:
GH_TOKEN: ${{ github.token }}
TAG: ${{ github.event.inputs.weights_tag }}
run: |
set -euo pipefail
mkdir -p weights
gh release download "$TAG" \
--repo "${{ github.repository }}" \
--pattern openai-google-source-v1.npz \
--dir weights
test -s weights/openai-google-source-v1.npz
- name: Verify artifact hash
run: |
set -euo pipefail
expected="$(jq -r .artifact_sha256 docs/image-source-hf/metrics.json)"
test "${#expected}" -eq 64
echo "$expected weights/openai-google-source-v1.npz" | sha256sum --check
- name: Publish to Hugging Face
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
SHA: ${{ github.sha }}
run: |
set -euo pipefail
if [ -z "${HF_TOKEN:-}" ]; then
echo "HF_TOKEN secret is not set" >&2
exit 1
fi
uv run --no-project --with huggingface-hub \
python scripts/publish_image_source_hf.py \
--src weights \
--message "Image-source classifier freeze (GitHub ${SHA:0:7})"
@@ -6,8 +6,9 @@ name: Publish photo classifier to Hugging Face
# Hub model: wiltodelta/raiw-photo-classify
# Secret: HF_TOKEN, write-role Hugging Face token (same secret Space sync uses).
#
# Weights are not in git. Full mode downloads them from a GitHub Release
# of this repository, then uploads with the card from docs/photo-classify-hf/.
# Weights are not in git. Full mode downloads the frozen PyTorch files from a
# GitHub Release, exports the vision-only ONNX graph, then uploads everything
# with the card from docs/photo-classify-hf/.
on:
workflow_dispatch:
@@ -16,7 +17,7 @@ on:
description: "Type publish"
required: true
mode:
description: "full = card plus weights from the release tag; card = README and operating-point only"
description: "full = card plus weights from the release tag; card = README, operating-point, and receipt gate only"
type: choice
options:
- full
@@ -52,6 +53,14 @@ jobs:
gh release download "$TAG" --repo "${{ github.repository }}" --dir weights
ls -l weights
- name: Export the vision-only ONNX graph
if: ${{ github.event.inputs.mode == 'full' }}
run: |
set -euo pipefail
uv run --extra classify --with onnx python scripts/export_photo_classify_onnx.py \
--checkpoint weights/clip-l-ft.pt \
--out weights/clip-l-ft-vision-fp32.onnx
- name: Publish to Hugging Face
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
@@ -0,0 +1,64 @@
name: Publish source classifier to Hugging Face
# Manual only. The derivative model weights live in a GitHub model-freeze
# release; the private training corpus and caches never enter this repository.
on:
workflow_dispatch:
inputs:
confirm:
description: "Type publish"
required: true
weights_tag:
description: "GitHub Release tag holding source-pipeline-mlp.npz"
required: true
default: source-classify-freeze-2026-09-16-r2
jobs:
publish:
if: ${{ github.event.inputs.confirm == 'publish' }}
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
steps:
- uses: actions/checkout@v7
- uses: astral-sh/setup-uv@v7
with:
python-version: "3.12"
- name: Download derivative model weights
env:
GH_TOKEN: ${{ github.token }}
TAG: ${{ github.event.inputs.weights_tag }}
run: |
set -euo pipefail
mkdir -p weights
gh release download "$TAG" \
--repo "${{ github.repository }}" \
--pattern source-pipeline-mlp.npz \
--dir weights
test -s weights/source-pipeline-mlp.npz
- name: Verify the frozen artifact
run: |
set -euo pipefail
expected="$(jq -r .artifact_sha256 docs/source-classify-hf/metrics.json)"
test "${#expected}" -eq 64
echo "$expected weights/source-pipeline-mlp.npz" | sha256sum --check
- name: Publish to Hugging Face
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
SHA: ${{ github.sha }}
run: |
set -euo pipefail
if [ -z "${HF_TOKEN:-}" ]; then
echo "HF_TOKEN secret is not set" >&2
exit 1
fi
uv run --no-project --with huggingface-hub \
python scripts/publish_source_classify_hf.py \
--src weights \
--message "Source-classify freeze (GitHub ${SHA:0:7})"
+3 -1
View File
@@ -10,6 +10,8 @@ permissions:
jobs:
publish:
# Model-only prereleases are not Python package releases.
if: ${{ startsWith(github.event.release.tag_name, 'v') && !github.event.release.prerelease }}
runs-on: ubuntu-latest
environment: pypi
steps:
@@ -38,4 +40,4 @@ jobs:
# is needed. uv's uploader also accepts Metadata-Version 2.5 -- the permanent
# fix that lets the hatchling pin be relaxed (see pyproject [build-system]).
- name: Publish to PyPI
run: uv publish
run: uv publish
+40
View File
@@ -0,0 +1,40 @@
name: Skill
# Fast path-filtered gate for agent-skill packaging changes. The full
# Tests workflow still runs on the same pushes; this answers skill-only
# pull requests in seconds by running just the deterministic skill suites.
on:
push:
branches: [main]
paths:
- "skills/**"
- ".claude-plugin/**"
- "docs/agent-skill.md"
- "tests/test_agent_skill.py"
- "tests/test_skill_evals.py"
pull_request:
paths:
- "skills/**"
- ".claude-plugin/**"
- "docs/agent-skill.md"
- "tests/test_agent_skill.py"
- "tests/test_skill_evals.py"
permissions:
contents: read
concurrency:
group: skill-${{ github.ref }}
cancel-in-progress: true
jobs:
skill:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: astral-sh/setup-uv@v7
- name: Sync dev environment
run: uv sync --frozen --extra dev
- name: Agent skill packaging tests
run: uv run pytest tests/test_agent_skill.py tests/test_skill_evals.py -q
+26 -1
View File
@@ -20,11 +20,28 @@ jobs:
- uses: astral-sh/setup-uv@v7
- name: Sync dev environment
run: uv sync --frozen --extra dev
- name: Check C2PA soft-binding registry snapshot
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: uv run python scripts/sync_c2pa_soft_bindings.py --check
- name: Ruff lint
run: uv run ruff check
- name: Ruff format check
run: uv run ruff format --check
security:
name: lockfile vulnerability scan
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v7
- uses: astral-sh/setup-uv@v7
with:
enable-cache: false
# Policy and triaged blocks: docs/development.md "Known security-gate blocks".
- name: Scan uv.lock with uv-secure
run: uvx uv-secure uv.lock
test:
name: test ${{ matrix.os }} py${{ matrix.python-version }}
runs-on: ${{ matrix.os }}
@@ -92,7 +109,15 @@ jobs:
with:
python-version: "3.12"
- name: Install ffmpeg
run: sudo apt-get update && sudo apt-get install --yes ffmpeg
# Third-party apt sources baked into the runner image (google-chrome et
# al.) can publish a broken index and fail `apt-get update`, while this
# job only needs ffmpeg from Ubuntu's own repositories: drop every
# non-Ubuntu source definition before updating.
run: |
sudo find /etc/apt/sources.list.d -mindepth 1 -maxdepth 1 \
! -name 'ubuntu.sources' -delete
sudo apt-get update
sudo apt-get install --yes ffmpeg
- name: Sync dev environment
run: uv sync --frozen --extra dev
- name: Run full-clip video test
+102 -24
View File
@@ -17,11 +17,10 @@ on:
workflows: ["Distribute on release"]
types: [completed]
# No step reads the GitHub token any more: the ComfyUI check moved to the
# registry's public API. Keep the job read-only so it cannot write on a repo
# whose default token permission is read/write.
# Read the triggering run's release identity while keeping all permissions read-only.
permissions:
contents: read
actions: read
jobs:
verify:
@@ -37,16 +36,55 @@ jobs:
SPACE_URL: https://wiltodelta-remove-ai-watermarks.hf.space
steps:
- name: Download distributed release identity
if: ${{ github.event_name == 'workflow_run' }}
uses: actions/download-artifact@v8
with:
pattern: distributed-release-*
path: distribution-identities
run-id: ${{ github.event.workflow_run.id }}
github-token: ${{ github.token }}
- name: Resolve target version
id: v
env:
EVENT_NAME: ${{ github.event_name }}
INPUT_VERSION: ${{ github.event.inputs.version }}
DISTRIBUTION_RUN_ID: ${{ github.event.workflow_run.id }}
DISTRIBUTION_RUN_ATTEMPT: ${{ github.event.workflow_run.run_attempt }}
run: |
set -euo pipefail
if [ -n "${{ github.event.inputs.version }}" ]; then
v="${{ github.event.inputs.version }}"
if [ "$EVENT_NAME" = "workflow_run" ]; then
v=$(python3 - <<'PYTHON'
import json, os
from pathlib import Path
identities = []
for path in Path("distribution-identities").rglob("release-identity.json"):
item = json.loads(path.read_text(encoding="utf-8"))
# download-artifact@v8 extracts the only matched artifact directly
# into the path, with no artifact-name directory; several matches
# each get one, so the attempt is name-checked only in that layout.
if path.parent.name.startswith('distributed-release-'):
assert path.parent.name == 'distributed-release-' + item['run_attempt'], "Artifact attempt mismatch"
identities.append(item)
assert identities, "No distributed release identity found"
assert all(item["run_id"] == os.environ["DISTRIBUTION_RUN_ID"] for item in identities), "Distribution run mismatch"
# Rerunning only failed jobs can reuse resolve from an earlier attempt.
eligible = [item for item in identities if
int(item["run_attempt"]) <= int(os.environ["DISTRIBUTION_RUN_ATTEMPT"])]
assert eligible, "No identity exists for this distribution attempt"
identity = max(eligible, key=lambda item: int(item["run_attempt"]))
print(identity["version"])
PYTHON
)
elif [ -n "$INPUT_VERSION" ]; then
v="$INPUT_VERSION"
else
v=$(curl -fsSL https://pypi.org/pypi/remove-ai-watermarks/json \
| python3 -c "import sys,json;print(json.load(sys.stdin)['info']['version'])")
fi
# Versions become URL components and step outputs. Reject malformed identities.
[[ "$v" =~ ^[0-9]+\.[0-9]+\.[0-9]+([a-zA-Z0-9.+-]*)$ ]]
echo "version=$v" >> "$GITHUB_OUTPUT"
echo "Verifying release $v"
@@ -106,28 +144,68 @@ jobs:
run: |
set -euo pipefail
echo "Checking the ComfyUI Registry node..."
# The registry publishes the node as JSON, including the dependency the
# published version declares. Read that directly instead of proxying
# through the sync workflow's run status: the node repo runs
# sync-library.yml on a daily schedule too, so a green nightly run says
# nothing about whether THIS release was ever synced.
node=$(curl -fsSL "https://api.comfy.org/nodes/remove-ai-watermarks")
echo "$node" | VERSION="$VERSION" python3 -c "
# Read the published versions directly. Do NOT proxy through the sync
# workflow's run status: the node repo runs that workflow on a daily
# schedule too, so a green nightly says nothing about THIS release.
#
# And read the versions list, not `latest_version` on the node object.
# The registry reviews a new upload before activating it, and
# `latest_version` reports only the active one -- so for the whole
# review window a correctly published version looks identical to one
# that was never uploaded. Those are opposite situations: one is a
# queue nobody here controls, the other is a broken release.
python3 -m pip install --quiet packaging
versions=$(curl -fsSL "https://api.comfy.org/nodes/remove-ai-watermarks/versions")
status=$(printf '%s' "$versions" | VERSION="$VERSION" python3 -c "
import json, os, sys
version = os.environ['VERSION']
latest = json.load(sys.stdin).get('latest_version') or {}
deps = latest.get('dependencies') or []
floor = f'>={version}'
if latest.get('status') != 'NodeVersionStatusActive':
sys.exit(f\"::error::ComfyUI node {latest.get('version')} is {latest.get('status')}, not active\")
if not any(floor in d for d in deps):
sys.exit(
f'::error::ComfyUI node {latest.get(\"version\")} does not require '
f'{version}; it declares {deps}'
from packaging.requirements import InvalidRequirement, Requirement
from packaging.utils import canonicalize_name
from packaging.version import Version
target = Version(version)
def requires_release(value):
try:
requirement = Requirement(value)
except InvalidRequirement:
return False
return (
canonicalize_name(requirement.name) == 'remove-ai-watermarks'
and requirement.url is None
and requirement.marker is None
and target in requirement.specifier
and any(s.operator == '>=' and Version(s.version) == target for s in requirement.specifier)
)
print(f' ComfyUI: node {latest[\"version\"]} requires {version}')
"
echo "- ComfyUI: registry node requires $VERSION" >> "$GITHUB_STEP_SUMMARY"
matches = [
v for v in json.load(sys.stdin)
if any(requires_release(d) for d in (v.get('dependencies') or []))
]
if not matches:
sys.exit(f'::error::No ComfyUI node version requires {version}; the sync never published one')
# Sort rather than trusting the response order: the endpoint happens to
# return newest-first today and nothing documents that it must.
newest = max(matches, key=lambda v: v.get('createdAt') or '')
print(f\"{newest.get('version')} {newest.get('status')}\")
")
node_version=${status%% *}
node_status=${status##* }
echo " ComfyUI: node $node_version requires $VERSION (status $node_status)"
case "$node_status" in
NodeVersionStatusActive)
echo "- ComfyUI: registry node $node_version requires $VERSION and is live" >> "$GITHUB_STEP_SUMMARY"
;;
NodeVersionStatusPending)
# Published and waiting on the registry's own review. Say exactly
# that in the summary rather than reporting it as live, and do not
# fail: an external moderation queue going red on every release
# trains everyone to ignore this workflow.
echo "::notice::ComfyUI node $node_version is published and awaiting registry review."
echo "- ComfyUI: registry node $node_version requires $VERSION, awaiting registry activation" >> "$GITHUB_STEP_SUMMARY"
;;
*)
echo "::error::ComfyUI node $node_version is $node_status"
exit 1
;;
esac
# -- HF Space -------------------------------------------------------------------
- name: Verify the HF Space serves this release
+15 -2
View File
@@ -9,6 +9,7 @@ tmp/
# Environment secrets
.env
.mcp.json
# OS files
.DS_Store
@@ -23,6 +24,17 @@ Thumbs.db
# Generated calibration inputs
data/synthid_corpus/refs/
# Local-only image pulls held for analysis. They are not publication-cleared and
# have no place in this repository's history. Until 2026-09-07 the only thing
# keeping them out of a commit was an untracked .gitignore inside the directory,
# which a fresh clone would not have.
data/spaces/
# Local-only research data: corpora, checkpoints and campaign reports, tens of
# gigabytes that no clone should be able to commit. The layout is described in
# data/research/README.md, which is local-only too.
data/research/
# Reference materials
_refs/
@@ -30,8 +42,8 @@ _refs/
yolov8n.pt
.coverage
# Local prototype and evaluation outputs (scripts/chroma_scrub_prototype.py and
# friends); the durable record of an oracle verdict lives under data/evaluations/
# Local prototype and evaluation outputs (research harnesses and probes); the
# durable record of an oracle verdict lives under data/evaluations/
out/
# Claude Code local settings
@@ -52,6 +64,7 @@ gfpgan/
# Local evaluation data
.local-eval/
/.oracle-slots.json
# Other harnesses' generated config. Claude's sync owns these; a committed copy
# is a second source nobody reconciles. Anchored so a same-named source file
+7 -4
View File
@@ -8,6 +8,7 @@ The project gives users control over provenance marks on content they generated
- Add visible templates only for AI-generation labels.
- Do not add stock, agency, or classifieds marks to `watermark_registry.py`.
- Do not add templates for regulator-defined disclosure icons (EU Code of Practice Annex 1).
- Keep `erase --region` generic and user-directed; do not build an automatic stock-watermark remover on it.
Full boundary and legal context: [`docs/legal-and-safety.md`](docs/legal-and-safety.md).
@@ -31,21 +32,23 @@ Optional features and installation groups are documented in [`docs/installation.
`maintain.sh` runs dependency freshness and security checks, Ruff, Pyright scoped to `src/`, and the parallel test suite. Full-project Pyright is not the project gate because the ML dependency graph can exhaust Node memory.
This gate does NOT pass `--ignore-unfixed` to uv-secure, so a transitive CVE with no released fix stops it before Ruff, Pyright and the tests ever run. That is deliberate and it is not a blocker: run those three separately and report them, per the triage in [`docs/development.md`](docs/development.md). A red gate here is not the same signal as a red gate in a sibling repository that does ignore unfixed findings.
The gate does not pass `--ignore-unfixed` to uv-secure, so a transitive CVE with no released fix stops it before Ruff, Pyright and the tests run. This matches the global rule: run and report those three separately. Scanner failures are fatal too. Triaged blockers and the recheck procedure: `Known security-gate blocks` in [`docs/development.md`](docs/development.md).
Command, gate, typing, and model-test invariants auto-load from [`.claude/rules/development.md`](.claude/rules/development.md). Environment recovery, CI behavior, and fixture policy live in [`docs/development.md`](docs/development.md).
Before a release, read [`docs/release-and-distribution.md`](docs/release-and-distribution.md). Treat the release as complete only after PyPI, Homebrew, the Hugging Face Space, and the ComfyUI Registry are verified; conda is not published. Keep the source-distribution public allowlist.
The published Agent Skill is [`skills/remove-ai-watermarks/`](skills/remove-ai-watermarks/). Update it in the same change as any CLI edit that alters commands, extras, exit codes, mark keys, or the intended-use boundary. Install and publish steps: [`docs/agent-skill.md`](docs/agent-skill.md).
## Module architecture
[`docs/module-internals.md`](docs/module-internals.md) is the canonical per-module map, including design decisions, thresholds, calibration history, incident records, and regression guards. Read the relevant section before changing a subsystem.
Research and current constraints are routed through [`docs/index.md`](docs/index.md), especially [`docs/known-limitations.md`](docs/known-limitations.md), [`docs/supported-signals.md`](docs/supported-signals.md), [`docs/synthid.md`](docs/synthid.md), and [`docs/watermarking-landscape.md`](docs/watermarking-landscape.md). Pixel photo classification is [`docs/photo-classify.md`](docs/photo-classify.md); it is not `identify`. Classifier research is split between general [`AI-generated image classifiers`](docs/ai-generated-image-classifiers.md) and [`SynthID source classifiers`](docs/synthid-classifiers.md). Other SynthID campaign logs are [`docs/synthid-detector-research.md`](docs/synthid-detector-research.md) and [`docs/synthid-removal-research.md`](docs/synthid-removal-research.md). The pre-split chronological archive is [`docs/synthid-detector-removal-plan.md`](docs/synthid-detector-removal-plan.md).
Research and current constraints are routed through [`docs/index.md`](docs/index.md), especially [`docs/known-limitations.md`](docs/known-limitations.md), [`docs/supported-signals.md`](docs/supported-signals.md), [`docs/synthid.md`](docs/synthid.md), and [`docs/watermarking-landscape.md`](docs/watermarking-landscape.md). Pixel photo classification is [`docs/photo-classify.md`](docs/photo-classify.md); the separate OpenAI/Google/unknown export-pipeline signal is [`docs/source-classify.md`](docs/source-classify.md). Neither runs inside `identify` or constitutes SynthID detection. Classifier research is split between general [`AI-generated image classifiers`](docs/ai-generated-image-classifiers.md) and [`SynthID source classifiers`](docs/synthid-classifiers.md). Other SynthID campaign logs are [`docs/synthid-detector-research.md`](docs/synthid-detector-research.md) and [`docs/synthid-removal-research.md`](docs/synthid-removal-research.md). The pre-split chronological archive is [`docs/synthid-detector-removal-plan.md`](docs/synthid-detector-removal-plan.md). The benchmark kernel and its pinned local oracles (AudioSeal, Perth, PixelSeal, VideoSeal) cover image, audio, and video watermark cohorts and research studies: [`docs/benchmark-kernel.md`](docs/benchmark-kernel.md).
## Data safety
Follow [`data/README.md`](data/README.md) for public fixture, calibration, oracle, and evaluation layout. Store each tracked binary once and keep generated evaluation outputs outside the repository.
Follow [`data/README.md`](data/README.md) for public fixture, calibration, oracle, and evaluation layout. Use only publication-cleared inputs in tracked data paths. Store each tracked binary once and keep generated evaluation outputs outside the repository.
## Rules and conventions
@@ -53,4 +56,4 @@ Topic-specific rules live in `.claude/rules/*.md` and are auto-loaded when match
| File | Covers |
|---|---|
| `development.md` | Command contracts, project gate, typing boundaries, model-adjacent tests, the docs-coverage seam, and the detection-path measurement rule |
| `development.md` | Command contracts, project gate, typing boundaries, model-adjacent tests, the docs-coverage and agent-skill parity seams, the detection-path measurement rule, and why cloud GPU harnesses stay out of `scripts/` |
+109 -56
View File
@@ -3,7 +3,7 @@
Remove AI provenance marks from images and video you generated yourself:
- known visible labels such as the Google Gemini sparkle watermark and vendor text marks;
- invisible pixel watermarks through diffusion regeneration;
- invisible pixel watermarks through direct local-format disruption or diffusion regeneration;
- C2PA, EXIF, XMP, IPTC, and related AI metadata.
Video support covers provenance identification, complete visible-plus-metadata
@@ -22,10 +22,14 @@ removal.
[![License](https://img.shields.io/pypi/l/remove-ai-watermarks?color=blue)](LICENSE)
[![Tests](https://github.com/wiltodelta/remove-ai-watermarks/actions/workflows/test.yml/badge.svg)](https://github.com/wiltodelta/remove-ai-watermarks/actions/workflows/test.yml)
[![Sponsor](https://img.shields.io/badge/Sponsor-GitHub-db61a2?logo=githubsponsors&logoColor=white)](https://github.com/sponsors/wiltodelta)
[![skills.sh](https://skills.sh/b/wiltodelta/remove-ai-watermarks)](https://skills.sh/wiltodelta/remove-ai-watermarks)
> This project is for lawful use on content you own. It does not target stock
> agency previews or other watermarks that protect third party paid content.
> See [scope, safety, and legal notes](docs/legal-and-safety.md).
> This is a research project on AI provenance signals, for lawful use on content
> you own. It is provided as is, without warranty or liability, and each user is
> responsible for how they use it. It does not target stock agency previews or
> other watermarks that protect third party paid content. Some jurisdictions,
> including China, restrict removing AI labels or supplying tools that do. See
> [scope, safety, and legal notes](docs/legal-and-safety.md).
## Choose what you want to do
@@ -33,7 +37,6 @@ removal.
| --- | --- | --- |
| Find provenance signals and watermarks | `identify` | No |
| Classify a photograph from pixels (opt-in, not provenance) | `classify` | No |
| Verify supported OpenAI SynthID from pixels with the official remote API | `verify-openai-synthid` | No |
| Remove known visible AI marks | `visible` | No |
| Erase a region you select | `erase` | No |
| Strip AI metadata | `metadata` | No |
@@ -42,14 +45,16 @@ removal.
| Strip AI metadata from video | `video metadata` | No |
| Remove a registered visible AI mark from video | `video visible` | No |
| Process a directory of videos | `video batch` | Depends on mode |
| Remove video SynthID with the certified VAE profile | `video invisible` | Recommended |
| Apply the calibrated video-pixel SynthID-removal profile | `video invisible` | Recommended |
| Regenerate an image to disrupt invisible watermarks | `invisible` | Required (CUDA) |
| Run visible, invisible, and metadata removal | `all` | Recommended |
| Process a directory | `batch` | Depends on mode |
Microsoft Paint and Photos InvisMark declarations are routed automatically to
pixel regeneration. The `all` command removes both the hidden pixel watermark and
its linked C2PA manifest; metadata stripping alone removes only the manifest.
its linked C2PA manifest; metadata stripping alone removes only the manifest. A
specialized Python API can inspect and disrupt the validated local
`Watermarker.dll` payload without diffusion.
## Installation modes
@@ -57,16 +62,17 @@ its linked C2PA manifest; metadata stripping alone removes only the manifest.
| --- | --- |
| Metadata inspection and stripping | `remove-ai-watermarks` |
| Photograph AI-versus-camera classification | `remove-ai-watermarks[classify]` |
| Official remote OpenAI SynthID verification | `remove-ai-watermarks[verify]` |
| OpenAI/Google/unknown source-export classification | `remove-ai-watermarks[source-classify]` |
| Visible detection and removal | `remove-ai-watermarks[visible]` |
| Visible video processing | `remove-ai-watermarks[video]` |
| Video SynthID removal | `remove-ai-watermarks[video,diffusion]` |
| Torch-free DWT-DCT detection | `remove-ai-watermarks[detect]` |
| Direct local Paint InvisMark disruption | `remove-ai-watermarks[pixels]` |
| Invisible image removal (needs CUDA) | `remove-ai-watermarks[qwen-zimage]` |
| Every production feature available on the active Python | `remove-ai-watermarks[all]` |
Lower-level and specialized extras include `pixels`, `heif`, `trustmark`,
`migan`, `lama`, and `diffusion`. The
`migan`, `lama`, `diffusion`, `classify-onnx`, and `source-classify`. The
[installation guide](docs/installation.md#feature-extras) documents their exact
dependency composition, Python compatibility, and model requirements.
@@ -94,24 +100,24 @@ remove-ai-watermarks classify image.png
Guide: [photo pixel classification](docs/photo-classify.md).
For a lightweight, abstaining OpenAI/Google/unknown source-export signal after
metadata removal, use the separate Python API. It is not a SynthID detector:
```python
import remove_ai_watermarks as raiw
result = raiw.classify_source("image.png")
print(result.label, result.reason)
```
Install `remove-ai-watermarks[source-classify]`. Guide:
[source-pipeline classification](docs/source-classify.md).
Signed provenance is the supported route for SynthID and `identify` reads it.
There is no local SynthID pixel detector in the package. Research on a
periodic lattice expert is in `scripts/synthid_runtime/` and
[synthid-detector-research.md](docs/synthid-detector-research.md).
For supported OpenAI images, the optional official verifier provides a pixel
watermark verdict:
```bash
uv tool install --force "remove-ai-watermarks[verify]"
remove-ai-watermarks verify-openai-synthid image.png --acknowledge-upload
```
The command removes AI provenance metadata from a temporary copy, verifies that
the decoded pixels are unchanged, uploads only that copy to OpenAI, and consumes
only the independent SynthID response. It never runs implicitly from `identify`.
An API key, endpoint access, and explicit upload acknowledgement are required.
For visible watermark removal, install the pixel dependencies:
```bash
@@ -147,6 +153,8 @@ inspection includes the native TC260 `AIGC` tag in
`moov.udta.meta.keys/ilst`, including a `moov` placed after the media payload,
plus the QuickTime-form `meta` variants Doubao's iOS export writes (a bare
`meta` box as a direct `moov` child, and a keyless `hdlr=mdir` metadata list).
Keyed `workflow` and `prompt` entries in the same metadata-list structure are
also detected and removed, including ComfyUI exports.
MKV and WebM inspection reads the normative
`Segment.Tags.Tag.SimpleTag` placement. AVI uses `LIST/INFO/AIGC`, while FLV
uses `script.onMetaData.AIGC`. The non-ISOBMFF formats are remuxed with stream
@@ -197,18 +205,21 @@ preserved through a timestamped in-memory NUT bridge instead of being flattened
to the average frame rate. Non-zero source start timestamps are retained
together with the copied audio offset. The default `--mark auto`
scans all providers in one decode pass and selects the first stable match in
the specificity order shown below. Pass an explicit mark to restrict detection
to one provider.
Sora covers the moving Sora 2 mascot and wordmark. Veo covers both the current
the specificity order shown below. Valid Google AI-video provenance breaks a
Sora/Veo cross-match in favor of Veo when the diamond is nested in the Sora
region or Veo persists through the complete clip while Sora does not.
Pass an explicit mark to restrict detection to one provider.
Sora covers the moving Sora 2 mascot, with or without the wordmark; mascot-only
matches also require the complete two-eye shape. Veo covers both the current
four-point diamond and the legacy `Veo` text. Seedance covers the fixed boxed
`AI` label, Dola covers the fixed `Dola AI` text, Hailuo AI covers the composite
`MINIMAX | hailuo AI` label, and Kling AI covers the bottom-right `KLING AI`
label with its version suffix. A completed encode is published atomically. No
output is written when no stable mark is found.
`MINIMAX | hailuo AI` label, and Kling AI covers the bottom-right `KLING AI` or
`KlingAI` label with its version suffix. A completed encode is published
atomically. No output is written when no stable mark is found.
HDR, PQ/HLG, and greater-than-8-bit inputs are rejected before encoding rather
than silently reduced through OpenCV's 8-bit BGR boundary.
Remove video SynthID:
Apply the calibrated video-pixel SynthID-removal profile:
```bash
uv tool install --force "remove-ai-watermarks[video,diffusion]"
@@ -221,10 +232,14 @@ completed encode atomically. The default `noise_std=0.15` profile passed both
the two-carrier calibration and a complete public eight-second Veo oracle
check. Google does not publish a local decoder, so a fresh provider check
remains useful for unusually important files or after provider changes, but it
is not a product result state.
is not a product result state. The CLI therefore describes video-pixel
regeneration instead of claiming a per-file SynthID verdict, and reports the
copied audio watermark as `UNVERIFIED`.
For invisible watermark removal, install the `qwen-zimage` extra. **An NVIDIA GPU
is required**: all profiles are CUDA-only, and there is no CPU or MPS fallback.
For invisible watermark removal through the CLI or high-level API, install the
`qwen-zimage` extra. **An NVIDIA GPU is required**: all diffusion profiles are
CUDA-only, and there is no CPU or MPS fallback. The specialized local Paint API
uses the `pixels` extra without CUDA.
```bash
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
@@ -277,8 +292,8 @@ ControlNet, followed by SAM-masked Z-Image repair of any detected face. The
alternative, `sdxl-zimage`, swaps the global stage for SDXL and keeps the same face
stage. A third profile, `chroma-zimage`, uses the Apache-2.0 Chroma1 global pass
with its own flat vendor floors; see `docs/chroma1-engine-research.md` for the
calibration. `--pipeline auto` picks chroma-zimage for OpenAI and Microsoft
provenance and qwen-zimage otherwise. All are CUDA only.
calibration. `--pipeline auto` picks sdxl-zimage for Google, chroma-zimage for
Microsoft, and qwen-zimage otherwise. All are CUDA only.
```bash
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
@@ -355,15 +370,22 @@ remove-ai-watermarks batch ./images --mode all
Visible mark support includes:
- Google Gemini and Nano Banana visible sparkle watermark;
- Doubao, the Jimeng wordmark and top-left `AI生成` pill, Qwen, Kling AI,
Yuanbao, Baidu, LiblibAI, and RunningHub labels;
- one calibrated Microsoft top-right white AI-badge variant;
- one calibrated Samsung Galaxy AI label variant.
- Doubao, the Jimeng wordmark and top-left `AI生成` layouts, Qwen, Kling AI,
Yuanbao, Dola, WorkBuddy, Baidu, LiblibAI's bottom-center wordmark and compact
top-left pill, and RunningHub labels;
- calibrated Microsoft top-right white and beige AI-badge layouts;
- calibrated Italian and Korean Samsung Galaxy AI disclosures;
- a generic brand-less bottom-right `AI生成` fallback for OS/gallery-level
AI-edit stamps that carry no vendor wordmark of their own (vivo and Xiaomi
Gallery confirmed); synthetic-only calibration, weaker evidence than the
vendor-tuned detectors above;
- the OpenArt wordmark, placed at the frame center rather than a corner;
uncalibrated (one confirmed real case, no captured corpus).
Metadata and provenance inspection covers C2PA, EXIF, XMP, IPTC, common
generator parameters, China TC260 AIGC labels, and several vendor specific
signals. Optional decoders add support for open DWT-DCT watermarks and Adobe
TrustMark.
generator parameters in image and video containers, China TC260 AIGC labels,
and several vendor specific signals. Optional decoders add support for open
DWT-DCT watermarks and Adobe TrustMark.
The exact support matrix, including important locale and detector limits, lives
in [supported signals](docs/supported-signals.md). The
@@ -380,13 +402,14 @@ Visible removal follows three steps:
Metadata removal uses format aware stripping. JPEG metadata removal preserves
the encoded image scan instead of recompressing it. Native MP4/MOV TC260 values
are blanked without changing box sizes or media offsets. Other supported
containers use their corresponding metadata path.
and keyed generation parameters are blanked without changing box sizes or
media offsets. Other supported containers use their corresponding metadata
path.
Invisible removal is different. It regenerates the image through a diffusion
pipeline to disrupt pixel and frequency domain watermarks. This changes the
image and cannot guarantee that a proprietary verifier will reject every
output.
Invisible removal through the CLI and high-level API regenerates the image through
a diffusion pipeline. Specialized callers can instead decode and disrupt the
positively identified local Microsoft Paint format with much smaller pixel changes.
Neither path can guarantee that every proprietary verifier will reject every output.
See [supported signals](docs/supported-signals.md) and
[known limitations](docs/known-limitations.md) for the full technical boundary.
@@ -401,15 +424,24 @@ import remove_ai_watermarks as raiw
result, removed = raiw.remove_visible("watermarked.png", "clean.png")
print(removed)
openai_synthid = raiw.verify_openai_synthid("image.png", acknowledge_upload=True)
print(openai_synthid.status)
report = raiw.remove_visible_detailed("watermarked.png", "clean.png")
print(report.status) # cleaned | partial | unvalidated | no_watermark
complete = raiw.remove_all("watermarked.png", "clean.png")
print(complete.visible_status, complete.visible_marks)
images = raiw.remove_batch("images", "images_clean", mode="visible")
for item in images.items:
print(item.source, item.visible_status, item.visible_marks)
provenance = raiw.identify_video("input.mp4")
report = raiw.inspect_video_metadata("input.mp4")
complete = raiw.remove_video_all("input.mp4", "clean.mp4")
print(complete.visual_invisible_action, complete.audio.watermark_status)
batch = raiw.remove_video_batch("videos", "videos_clean")
cleaned = raiw.remove_video_metadata("input.mp4")
synthid_cleaned = raiw.remove_video_invisible("input.mp4", "synthid_clean.mp4")
print(synthid_cleaned.visual_invisible_action, synthid_cleaned.audio.watermark_status)
visible = raiw.remove_video_visible("input.mp4", "clean.mp4")
print(visible.mark)
veo = raiw.remove_video_visible("veo.mp4", "veo_clean.mp4", mark="veo")
@@ -429,6 +461,24 @@ See the [Python API guide](docs/python-api.md) for visible removal, the full
`remove_all` and `remove_batch` pipeline, provenance inspection, metadata
stripping, and diffusion usage.
## Agent skill
Coding agents can install the published skill and drive this CLI:
```bash
npx skills add wiltodelta/remove-ai-watermarks
```
Claude Code can add this repository as a plugin marketplace:
```text
/plugin marketplace add wiltodelta/remove-ai-watermarks
/plugin install remove-ai-watermarks@remove-ai-watermarks
```
The skill refuses stock-agency and other third-party paid-asset marks. See
[the agent skill guide](docs/agent-skill.md).
## ComfyUI
The separate
@@ -446,7 +496,7 @@ invisible removal.
detail.
- Visible video removal recognizes the moving Sora 2 wordmark, the current Veo
diamond plus legacy `Veo` text, the Seedance boxed `AI` label, and the fixed
Doubao, Dola, Hailuo AI, and Kling AI labels. It does not recognize the older Sora Turbo
Doubao, Dola, Hailuo AI, Vidu, and Kling AI labels. It does not recognize the older Sora Turbo
corner swirl or unregistered layouts from those providers.
The classical OpenCV backend can smear structured backgrounds; use MI-GAN or
LaMa when recovery quality matters.
@@ -454,9 +504,10 @@ invisible removal.
The shipped profile is oracle-certified, but no public local decoder can
certify an arbitrary output at runtime. Recheck unusually important outputs
after provider changes.
- Invisible-watermark removal requires CUDA. All profiles refuse any other
device at construction rather than falling back to one that cannot run them.
Visible removal, metadata stripping and `identify` still run anywhere.
- Invisible-watermark removal through the CLI and high-level API requires CUDA.
All diffusion profiles refuse any other device at construction rather than
falling back to one that cannot run them. The specialized local Paint API,
visible removal, metadata stripping and `identify` run without CUDA.
- Provider watermark systems can change. Validate important outputs with the
provider's own verifier when one is available.
@@ -470,8 +521,8 @@ uv run --extra video --extra diffusion python scripts/video_synthid_sweep.py inp
```
To score what an output actually cost, use `scripts/video_fidelity_probe.py`:
the engine's own PSNR is measured before the resize and the encode, so only the
probe sees the delivered picture.
the engine's own PSNR is measured after the input resize and before final
encoding, so only the probe sees the delivered picture.
```bash
uv run --extra video python scripts/video_fidelity_probe.py input.mp4 input_clean.mp4
@@ -504,6 +555,7 @@ Start with the [documentation index](docs/index.md).
- [Scope, safety, and legal notes](docs/legal-and-safety.md)
- [Module internals](docs/module-internals.md)
- [Release and distribution](docs/release-and-distribution.md)
- [Agent skill](docs/agent-skill.md)
Research notes and historical experiments are listed separately in the
[documentation index](docs/index.md). They explain past decisions but do not
@@ -519,7 +571,8 @@ bash maintain.sh
```
See [module internals](docs/module-internals.md) before changing a subsystem
with documented invariants.
with documented invariants. Report vulnerabilities as described in
[SECURITY.md](SECURITY.md).
## License
+25
View File
@@ -0,0 +1,25 @@
# Security policy
## Supported versions
Security fixes go into the latest release. Older releases are not patched;
upgrade through the channel you installed from (PyPI or Homebrew).
## Reporting a vulnerability
Report vulnerabilities privately through
[GitHub private vulnerability reporting](https://github.com/wiltodelta/remove-ai-watermarks/security/advisories/new).
Do not open a public issue for an unpatched vulnerability. Include the affected
version, the command or API call, and a minimal input that reproduces it.
Vulnerabilities in a dependency, such as a CVE in a package from `uv.lock`, can
be reported as a regular issue when the advisory is already public.
## Dependency vulnerabilities
Dependabot raises alerts and update pull requests for `uv.lock` and GitHub
Actions, and CI scans `uv.lock` with
[`uv-secure`](https://github.com/owenlamont/uv-secure) on every push and pull
request. A vulnerable dependency is upgraded to a fixed release; advisories
without one are tracked under
[Known security-gate blocks](docs/development.md#known-security-gate-blocks).
+27 -10
View File
@@ -7,8 +7,14 @@ data/
fixtures/
provenance/ Real format and provenance fixtures used by tests
(source records live in fixtures/README.md)
visible/ Synthetic per-mark example gallery (one committed example per
registered visible mark; see fixtures/visible/README.md)
visible/ Per-mark example gallery: one synthetic canonical example per
registered mark plus documented provider originals where available
(see fixtures/visible/README.md)
synthetic/ Frozen synthetic renders that unit tests stamp onto frames, each
produced by the script named in the test that loads it
captures/
2026-09/ Provider outputs from the maintainer's own accounts, stored as
served, indexed by manifest.csv (see its README)
calibration/
<vendor>/ Minimal controlled inputs needed to rebuild detector assets
synthid/
@@ -18,19 +24,22 @@ data/
Reusable full-pipeline evaluation selection
evaluations/
engine-selection/
Prompt-matched AI content matrix for deterministic auto-engine research
Prompt-matched, publication-cleared AI content matrix for
deterministic auto-engine and watermark benchmark research
fidelity/ Evaluation instructions and hand-verified ground truth
video-synthid-oracle.csv
Reproducible full-clip Gemini SynthID verdicts
```
[`image-watermark-examples.csv`](image-watermark-examples.csv) and
[`video-watermark-examples.csv`](video-watermark-examples.csv) are the coverage
indexes for real carriers. They point to each canonical binary instead of
duplicating it and record unsupported gaps and source-specific licensing.
## Storage rules
Large local-only research data (the 211 GB Spaces corpus and the frozen
checkpoints, research corpora, and campaign reports) lives under gitignored
paths documented in `data/research/README.md` (ignored via `.git/info/exclude`,
never committed). Worktree `.local-eval/` directories are scratch copies; the
durable store is `data/research/` in the main checkout.
Use only publication-cleared inputs in tracked data paths. Keep generated
evaluation outputs outside the repository.
1. Store each binary image once. Evaluation manifests and documentation point
to its canonical location.
@@ -42,12 +51,20 @@ durable store is `data/research/` in the main checkout.
commands, hashes, and oracle verdicts instead of committing another corpus
copy. A small curated before-and-after example may live in `docs/images/`
when it is part of the public documentation.
Local benchmark manifests may point to these canonical inputs or to an
untracked corpus; every artifact and reference must carry its SHA-256 digest.
`evaluations/engine-selection/` is the bounded exception for ORIGINAL study
inputs: its manifest tracks a prompt-matched OpenAI/Meta content matrix, not
generated model outputs or watermark-oracle evidence.
6. Keep third-party fixture license notices in `licenses/`, outside directories
6. Put provider captures that no test reads in `captures/<yyyy-mm>/`, with a
`manifest.csv` row per capture. A capture that is also a fixture is stored
once, in `fixtures/`, and its manifest row points there. Request a new
capture only for a service x vendor pair not yet captured, or for a
service's own model: another version of a known pair adds no provenance
evidence.
7. Keep third-party fixture license notices in `licenses/`, outside directories
that tests enumerate as media inputs.
7. Runtime detector assets belong in `src/remove_ai_watermarks/assets/`.
8. Runtime detector assets belong in `src/remove_ai_watermarks/assets/`.
Unregistered research candidates belong in
`scripts/assets/visible-mark-candidates/` so they are not shipped in the
wheel.
+4 -3
View File
@@ -2,9 +2,10 @@
> **Status (built 2026-06-05):** flat black and gray Samsung Galaxy AI captures
> were obtained and the detection asset was solved. The asset now supports
> detection and mask geometry only. Current removal localizes the wordmark and
> uses the shared fill backend. The text below records the capture plan and its
> locale and resolution limits.
> detection and detector-aligned alpha-mask geometry. Current removal aligns the
> solved alpha to the detector's winning template box and uses the shared fill
> backend. The text below records the capture plan and its locale and resolution
> limits.
Goal: capture the Samsung Galaxy AI "✦ Contenuti generati dall'AI" visible wordmark
over known flat backgrounds so we can rebuild and validate the silhouette used
+44
View File
@@ -0,0 +1,44 @@
# Provider captures, September 2026
Original outputs downloaded from the maintainer's own accounts between
2026-09-23 and 2026-09-25 and cleared for publication by the maintainer. The
services are the Gemini API, xAI Grok, Vidu, Wan, Apple Image Playground,
YouTube (re-encodes of our own uploads), and the paid Higgsfield, Runway and
Dreamina plans. Every prompt is a neutral coffee-mug scene; Higgsfield
Genjutsu and the video-edit samples take our own Higgsfield outputs as input.
Files are stored byte-for-byte as served. Nothing is re-encoded, because the
provenance under study (C2PA, TC260 labels, `hf-job-id` chunks) lives in the
container bytes.
`manifest.csv` lists every capture once, including the ones that are already
test fixtures: those rows point at their `data/fixtures/` path instead of a
second copy (`stored` is `existing`). Columns:
| Column | Meaning |
| --- | --- |
| `sha256` | Digest of the stored file |
| `path` | Canonical repository path |
| `stored` | `new` for files in this directory, `existing` for a fixture already tracked |
| `service` | Service the file was downloaded from |
| `source_name` | Local capture name, which names the model |
| `captured` | Capture date |
| `ai_from_metadata`, `platform` | `identify` (images) or `identify_video` (videos) verdict from metadata only, at capture time |
What each service does to the upstream provenance, and the per-model
findings, are summarized in the live-captures table of
[`docs/watermarking-landscape.md`](../../../docs/watermarking-landscape.md).
The `google-photos/` files are the maintainer's own iPhone photos edited in
Google Photos on iOS (an Ask edit replacing a dog with a cat, an eraser edit, and
two other AI edits), published unmodified with their EXIF GPS position by the
maintainer's decision on 2026-09-25. Google's checker in Gemini found SynthID in
all four with the metadata stripped. The Apple Clean Up photo is not stored.
The `google-photos-web/` files are a negative control from the Google Photos web
editor (2026-09-26): one of the maintainer's photos, downloaded untouched and after
Enhance, Sky (Vivid) and a square crop. The web editor offers no generative tool and
writes no Content Credentials, so none of the four carries C2PA, IPTC or an AI
credit, and Google's checker in Gemini found no SynthID in any of them (three
explicit negatives; the crop drew a reply that did not name SynthID and is recorded
as indeterminate).
- License: repository license; generated and explicitly cleared by the maintainer
Binary file not shown.

After

Width:  |  Height:  |  Size: 6.2 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.8 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 7.3 MiB

Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.5 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 395 KiB

Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 335 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 407 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 679 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 636 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 603 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 627 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 715 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 529 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 420 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 146 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 183 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 120 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 117 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.3 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.6 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.9 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 691 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 182 KiB

Binary file not shown.
Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 945 KiB

Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.0 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.7 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.9 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.7 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 MiB

Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.6 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.3 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.2 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.5 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.5 MiB

Binary file not shown.
Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 4.6 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.4 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.9 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.7 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.2 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.2 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.6 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.9 MiB

Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 MiB

Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Loaded 100 of 517 files, more files were not shown because too many files have changed in this diff. Show more