Victor Kuznetsov
464fd142ea
Extend the development-only benchmark family from images to audio and video, and close four research directions from the watermark-landscape map with local, revision-pinned evidence. Benchmark schema: media_type audio and video rows, audioseal and videoseal adapters with media-checked adapter binding, sample-domain audio fidelity (SNR with unbounded_identical), frame-domain video fidelity (mean PSNR, changed-frame fraction), and per-frame temporal evidence in videoseal detection records. The report accepts all three media types and carries SNR columns. Audio: shared pinned AudioSeal oracle, synthetic and system-voice speech cohorts with matched negatives, codec, noise, removed, and forged arms, and the provenance experiment proving visual-only video cleaning leaves the audio bitstream and watermark verdict invariant. Video: TorchScript-only VideoSeal oracle (the pip package is unusable on macOS arm64 and its loader resolves paths relative to cwd), video cohort with crf/scale/fps attacks and four-state arms, temporal evaluation measuring the crf decay curve, the frame-aggregation matrix (computed locally after finding the TorchScript build ignores the aggregation argument), and real provider clips as carriers. Forgery: double-embedding study showing the last writer wins completely under both oracles, scoped remover-trace probe, and the forged/ removed cohort arms encoding each scheme's correct expected verdict (presence rule for audioseal, message rule for videoseal). C2PA resolution: measured the five vendors' resolver contracts and the C2PA 2.2 18.10.5 privacy gradient; product decision recorded that this tool never resolves over the network. A registry guard pins the published resolution-API set. Gate: ruff and pyright src/ clean, 2114 passed / 3 skipped; maintain.sh stops only on the known unfixed lightning 2.6.5 CVEs, pre-existing.
4.8 KiB
Remove AI Watermarks
You are a principal Python engineer maintaining a CLI tool and library for removing visible and invisible AI provenance watermarks.
Scope and non-goals
The project gives users control over provenance marks on content they generated or edited themselves. It does not automatically remove stock-agency, marketplace, classifieds, tiled-preview, or other marks that protect a third party's paid or copyrighted asset.
- Add visible templates only for AI-generation labels.
- Do not add stock, agency, or classifieds marks to
watermark_registry.py. - Keep
erase --regiongeneric and user-directed; do not build an automatic stock-watermark remover on it.
Full boundary and legal context: docs/legal-and-safety.md.
How to run
uv run remove-ai-watermarks --help
bash maintain.sh
Run uv from the repository root. Command selection, options, defaults, and examples live in docs/cli.md. Before changing command routing, no-signal behavior, or exit codes, read the command-line section of docs/module-internals.md.
Configuration
GPU and ML modules are optional. Guard their imports with is_available().
Optional features and installation groups are documented in docs/installation.md. Model-running paths may use availability tests, while pure helpers in ML-adjacent modules must remain unit-tested without downloads.
Test and lint
maintain.sh runs dependency freshness and security checks, Ruff, Pyright scoped to src/, and the parallel test suite. Full-project Pyright is not the project gate because the ML dependency graph can exhaust Node memory.
This gate does NOT pass --ignore-unfixed to uv-secure, so a transitive CVE with no released fix stops it before Ruff, Pyright and the tests ever run. That is deliberate and it is not a blocker: run those three separately and report them, per the triage in docs/development.md. A red gate here is not the same signal as a red gate in a sibling repository that does ignore unfixed findings.
Command, gate, typing, and model-test invariants auto-load from .claude/rules/development.md. Environment recovery, CI behavior, and fixture policy live in docs/development.md.
Before a release, read docs/release-and-distribution.md. Treat the release as complete only after PyPI, Homebrew, the Hugging Face Space, and the ComfyUI Registry are verified; conda is not published. Keep the source-distribution public allowlist.
The published Agent Skill is skills/remove-ai-watermarks/. Update it in the same change as any CLI edit that alters commands, extras, exit codes, mark keys, or the intended-use boundary. Install and publish steps: docs/agent-skill.md.
Module architecture
docs/module-internals.md is the canonical per-module map, including design decisions, thresholds, calibration history, incident records, and regression guards. Read the relevant section before changing a subsystem.
Research and current constraints are routed through docs/index.md, especially docs/known-limitations.md, docs/supported-signals.md, docs/synthid.md, and docs/watermarking-landscape.md. Pixel photo classification is docs/photo-classify.md; it is not identify. Classifier research is split between general AI-generated image classifiers and SynthID source classifiers. Other SynthID campaign logs are docs/synthid-detector-research.md and docs/synthid-removal-research.md. The pre-split chronological archive is docs/synthid-detector-removal-plan.md. The benchmark kernel and its pinned local oracles (AudioSeal, VideoSeal) cover the audio/video watermark cohorts and research studies: docs/benchmark-kernel.md.
Data safety
Follow data/README.md for public fixture, calibration, oracle, and evaluation layout. Store each tracked binary once and keep generated evaluation outputs outside the repository.
Rules and conventions
Topic-specific rules live in .claude/rules/*.md and are auto-loaded when matching files are touched.
| File | Covers |
|---|---|
development.md |
Command contracts, project gate, typing boundaries, model-adjacent tests, the docs-coverage and agent-skill parity seams, and the detection-path measurement rule |