diff --git a/.claude/rules/development.md b/.claude/rules/development.md index 30c49d8..49c1455 100644 --- a/.claude/rules/development.md +++ b/.claude/rules/development.md @@ -13,6 +13,10 @@ Exit-code and no-signal behavior is a public contract. Read the command-line sec `tests/test_agent_skill.py` enforces that by comparing the skill's prose against the live Click tree. Guard the SET, not the instances: per-flag comparisons only check what the skill already documents, so `--vendor`, `--pipeline auto` and then the whole `classify` and `verify-openai-synthid` commands each shipped with a green suite. `test_every_cli_command_is_named_somewhere_in_the_skill` closes that; raise `MIN_CLI_VERSION` in the skill's probe whenever a reference names a command, flag or value the previous release lacked, or the probe tells an agent its CLI is current right before it types something that build rejects. +Provider verification services and provenance APIs are development-only oracles. Keep +their SDKs in the development extra and their adapters out of the installed CLI, +top-level Python API, and runtime detection/removal pipelines. + Do not add an option whose only outcome is an error. Model id, step count and CFG are fixed by the profile, so none of them is a parameter of the CLI, `InvisibleEngine`, or `WatermarkRemover` -- they were accepted-then-rejected for a while, which moved the failure several frames below the caller and advertised choices the pinned stack cannot honor. If a value cannot vary, delete the knob rather than validating it. `device` is the deliberate exception and stays a library parameter: `None`/`"auto"` detect, `"cuda"` pins without detecting, and everything else raises at construction. On the image path it is not a CLI option, because the only value a user could usefully type is the one auto-detection already finds. The video SynthID commands (`video invisible`, `video all`, `video batch`) do expose `--device`, whose VAE runs on cuda, mps, or cpu, so a user can usefully pick one. `test_device_exists_exactly_where_the_skill_says_it_does` pins that split; do not read this paragraph as licence to delete the video flag. diff --git a/README.md b/README.md index f752c18..0ea542a 100644 --- a/README.md +++ b/README.md @@ -34,7 +34,6 @@ removal. | --- | --- | --- | | Find provenance signals and watermarks | `identify` | No | | Classify a photograph from pixels (opt-in, not provenance) | `classify` | No | -| Verify supported OpenAI SynthID from pixels with the official remote API | `verify-openai-synthid` | No | | Remove known visible AI marks | `visible` | No | | Erase a region you select | `erase` | No | | Strip AI metadata | `metadata` | No | @@ -43,7 +42,7 @@ removal. | Strip AI metadata from video | `video metadata` | No | | Remove a registered visible AI mark from video | `video visible` | No | | Process a directory of videos | `video batch` | Depends on mode | -| Remove video SynthID with the certified VAE profile | `video invisible` | Recommended | +| Apply the calibrated video-pixel SynthID-removal profile | `video invisible` | Recommended | | Regenerate an image to disrupt invisible watermarks | `invisible` | Required (CUDA) | | Run visible, invisible, and metadata removal | `all` | Recommended | | Process a directory | `batch` | Depends on mode | @@ -58,7 +57,6 @@ its linked C2PA manifest; metadata stripping alone removes only the manifest. | --- | --- | | Metadata inspection and stripping | `remove-ai-watermarks` | | Photograph AI-versus-camera classification | `remove-ai-watermarks[classify]` | -| Official remote OpenAI SynthID verification | `remove-ai-watermarks[verify]` | | Visible detection and removal | `remove-ai-watermarks[visible]` | | Visible video processing | `remove-ai-watermarks[video]` | | Video SynthID removal | `remove-ai-watermarks[video,diffusion]` | @@ -100,19 +98,6 @@ There is no local SynthID pixel detector in the package. Research on a periodic lattice expert is in `scripts/synthid_runtime/` and [synthid-detector-research.md](docs/synthid-detector-research.md). -For supported OpenAI images, the optional official verifier provides a pixel -watermark verdict: - -```bash -uv tool install --force "remove-ai-watermarks[verify]" -remove-ai-watermarks verify-openai-synthid image.png --acknowledge-upload -``` - -The command removes AI provenance metadata from a temporary copy, verifies that -the decoded pixels are unchanged, uploads only that copy to OpenAI, and consumes -only the independent SynthID response. It never runs implicitly from `identify`. -An API key, endpoint access, and explicit upload acknowledgement are required. - For visible watermark removal, install the pixel dependencies: ```bash @@ -209,7 +194,7 @@ atomically. No output is written when no stable mark is found. HDR, PQ/HLG, and greater-than-8-bit inputs are rejected before encoding rather than silently reduced through OpenCV's 8-bit BGR boundary. -Remove video SynthID: +Apply the calibrated video-pixel SynthID-removal profile: ```bash uv tool install --force "remove-ai-watermarks[video,diffusion]" @@ -222,7 +207,9 @@ completed encode atomically. The default `noise_std=0.15` profile passed both the two-carrier calibration and a complete public eight-second Veo oracle check. Google does not publish a local decoder, so a fresh provider check remains useful for unusually important files or after provider changes, but it -is not a product result state. +is not a product result state. The CLI therefore describes video-pixel +regeneration instead of claiming a per-file SynthID verdict, and reports the +copied audio watermark as `UNVERIFIED`. For invisible watermark removal, install the `qwen-zimage` extra. **An NVIDIA GPU is required**: all profiles are CUDA-only, and there is no CPU or MPS fallback. @@ -412,15 +399,14 @@ images = raiw.remove_batch("images", "images_clean", mode="visible") for item in images.items: print(item.source, item.visible_status, item.visible_marks) -openai_synthid = raiw.verify_openai_synthid("image.png", acknowledge_upload=True) -print(openai_synthid.status) - provenance = raiw.identify_video("input.mp4") report = raiw.inspect_video_metadata("input.mp4") complete = raiw.remove_video_all("input.mp4", "clean.mp4") +print(complete.visual_invisible_action, complete.audio.watermark_status) batch = raiw.remove_video_batch("videos", "videos_clean") cleaned = raiw.remove_video_metadata("input.mp4") synthid_cleaned = raiw.remove_video_invisible("input.mp4", "synthid_clean.mp4") +print(synthid_cleaned.visual_invisible_action, synthid_cleaned.audio.watermark_status) visible = raiw.remove_video_visible("input.mp4", "clean.mp4") print(visible.mark) veo = raiw.remove_video_visible("veo.mp4", "veo_clean.mp4", mark="veo") diff --git a/docs/ai-generated-image-classifiers.md b/docs/ai-generated-image-classifiers.md index 3cb21c2..eea6999 100644 --- a/docs/ai-generated-image-classifiers.md +++ b/docs/ai-generated-image-classifiers.md @@ -1497,4 +1497,4 @@ task, not a watermark task. Adjacent papers: | Wang, Wang, Zhang, Owens, Efros, [arXiv:1912.11035](https://arxiv.org/abs/1912.11035) (CVPR 2020, CNNDetect) | Classifier on ProGAN + JPEG/crop aug transfers to many CNNs | The "one generator is enough" claim. Ojha is the correction once diffusion exists | | Wang et al., DIRE, [arXiv:2303.09295](https://arxiv.org/abs/2303.09295) (ICCV 2023) | Reconstruction error under a frozen diffusion model | SDXL float32 at 512. VAE RMS: camera 11.84, photo edit 9.89, s1 9.15, gray stamp 1.24. DDIM DIRE RMS: camera 33.0, s1 31.5, photo 30.9, gray 2.40. Texture rank, not a payload. Float16 DDIM NaN'd on MPS | -They do not substitute for `verify-openai-synthid`. +They do not substitute for a provider oracle during development. diff --git a/docs/cli.md b/docs/cli.md index a077e5b..97882f5 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -15,7 +15,6 @@ defaults. This page focuses on choosing the right command. | --- | --- | | `metadata` and metadata-only `identify` | Default package | | `classify` | `remove-ai-watermarks[classify]` | -| `verify-openai-synthid` | `remove-ai-watermarks[verify]`, API access, and `OPENAI_API_KEY` | | Visible signals in `identify` | `remove-ai-watermarks[visible]` (`pixels` is the minimal runtime) | | Open DWT-DCT signals in `identify` | `remove-ai-watermarks[detect]` | | Adobe TrustMark signals in `identify` on Python 3.11-3.12 | `remove-ai-watermarks[trustmark]` | @@ -70,41 +69,6 @@ Despite the historical option name, `--no-visible` skips all pixel detectors, including visible marks, open DWT-DCT, and TrustMark. Metadata inspection still runs. -## Verify OpenAI SynthID from pixels - -```bash -uv tool install --force "remove-ai-watermarks[verify]" -remove-ai-watermarks verify-openai-synthid image.png --acknowledge-upload -remove-ai-watermarks verify-openai-synthid image.png --acknowledge-upload --json -``` - -This is an explicit remote check against OpenAI's official Content Provenance -API, not the incomplete local OpenAI carrier research model. Before upload, the -command writes a temporary copy with AI provenance metadata removed and aborts -unless the decoded RGBA pixels are identical to the source. It then reads only -the API's independent `synthid` entry; a C2PA-only response cannot become a -SynthID detection. The source is never modified. - -The API supports PNG, JPEG, and WebP files up to 50 MiB. The command requires -`OPENAI_API_KEY` and an organization with endpoint access. Because the sanitized -raster is uploaded to OpenAI and the endpoint is not eligible for Zero Data -Retention, `--acknowledge-upload` is mandatory. This command is never called by -`identify`. `not_detected` means only that OpenAI's verifier did not recognize a -supported watermark in this file; it is not proof of human authorship. - -The built-in client bounds the request at 120 seconds and disables automatic -SDK retries, so one acknowledgement cannot silently upload the media multiple -times. A timeout, disconnect, malformed response, access failure, or rate limit -is an error, never a negative watermark verdict. API failures expose status, -error code, request id, `Retry-After`, and whether an explicit caller-controlled -retry is appropriate through `OpenAIProvenanceError`; the verifier itself never -retries an upload. -The JSON result uses the same provider-scope, backend, pixel-preservation, and -metadata-use audit fields as `verify_openai_synthid`. - -The Python API enforces the same boundary with the required explicit intent -flag `verify_openai_synthid(path, acknowledge_upload=True)`. - ## Classify a photograph from pixels This is not provenance. `identify` never starts it, including after a @@ -367,7 +331,7 @@ different container extension. Visible video labels and invisible video watermarks are not handled by this command. -## Remove video SynthID +## Apply the video-pixel SynthID-removal profile ```bash uv tool install --force "remove-ai-watermarks[video,diffusion]" @@ -405,6 +369,12 @@ short-clip calibration and the complete public eight-second Veo carrier in the calibration certifies the shipped operating point; the paired check above is an optional runtime audit, not a separate result state. +On completion the command reports that video pixels were regenerated with the +calibrated profile, not that this file received a fresh negative verdict. It +separately prints `Audio watermark: UNVERIFIED` because source audio is copied +unchanged when present. `video all` and `video batch` use the same wording and +never fold the audio status into the visual action. + ## Remove a supported visible video mark ```bash @@ -435,7 +405,8 @@ provider. The video stream is transcoded and the complete original audio stream is copied without truncating an audio tail that extends beyond the final video -frame. The encoder probes the source stream and preserves supported 8-bit +frame. The CLI reports that copied audio as watermark-unverified. The encoder +probes the source stream and preserves supported 8-bit chroma sampling, color range/matrix/transfer/primaries tags, and MP4/MOV track timescale. For a variable-frame-rate source, decoded PTS are carried through a timestamped in-memory NUT bridge so the output retains the source frame diff --git a/docs/installation.md b/docs/installation.md index e10148c..70ff60f 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -126,7 +126,6 @@ application actually uses: | `detect` | Open DWT-DCT detection for Stable Diffusion, SDXL, and FLUX | `pixels`, PyWavelets | No | | `trustmark` | Adobe TrustMark detection on Python 3.11-3.12 | trustmark | Yes | | `classify` | Metadata-free photo AI-versus-camera classifier plus gated provider | `pixels`, Torch, Transformers | Yes | -| `verify` | Official remote OpenAI SynthID verification | OpenAI SDK | No | | `diffusion` | Torch and Diffusers runtime; video SynthID regeneration | `pixels`, Torch, Diffusers | Yes | | `migan` | MI-GAN ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch | | `lama` | big-LaMa ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch | @@ -134,7 +133,7 @@ application actually uses: | `text-restoration` | Opt-in verified profile-VAE glyph restoration | `qwen-zimage`, `lama` | Yes | | `text-draft` | Draft OCR proposals for operator verification | PaddleOCR, PaddlePaddle | Model download, no Torch | | `all` | Every production feature available on the active Python | All compatible rows above | Yes | -| `dev` | Tests, linting, typing, and upstream parity checks | `video`, `detect`, upstream invisible-watermark | Yes, for parity tests | +| `dev` | Tests, linting, typing, upstream parity checks, and development oracles | `video`, `detect`, OpenAI SDK, upstream invisible-watermark | Yes, for parity tests | Dependency composition: @@ -153,10 +152,9 @@ flowchart LR heif trustmark classify --> pixels - verify ``` -`heif`, `trustmark`, `verify`, and `text-draft` are independent branches. Combine them +`heif`, `trustmark`, and `text-draft` are independent branches. Combine them explicitly with another feature when required. `text-draft` is excluded from `all` because it proposes unverified OCR annotations and is not a production removal path. TrustMark requires NumPy 1.x, which has no @@ -179,9 +177,6 @@ uv tool install --force "remove-ai-watermarks[video]" # DWT-DCT and TrustMark detection without diffusion removal uv tool install --force "remove-ai-watermarks[detect,trustmark]" -# Official OpenAI SynthID verification -uv tool install --force "remove-ai-watermarks[verify]" - # Every production capability compatible with this Python uv tool install --force "remove-ai-watermarks[all]" @@ -194,13 +189,6 @@ do not install libheif. `detect` uses the in-tree torch-free decoder and does not install the upstream `invisible-watermark` package. Optional models download their weights on first use. -The `verify` extra makes an explicit remote request. The -`verify-openai-synthid` command first removes AI provenance metadata from a -temporary copy, checks that its decoded pixels are unchanged, and then uploads -that copy to OpenAI. It needs `OPENAI_API_KEY`; the command never runs from -`identify` and refuses to upload without `--acknowledge-upload`. The Python API -requires the equivalent explicit `acknowledge_upload=True` argument. - The old `gpu` and `remove` aliases are intentionally not provided. Use `diffusion` and `visible` respectively. diff --git a/docs/known-limitations.md b/docs/known-limitations.md index 03855e8..8df2c4f 100644 --- a/docs/known-limitations.md +++ b/docs/known-limitations.md @@ -134,8 +134,8 @@ rates are kept in Google does not publish the proprietary SynthID payload decoder, and the package does not ship one. Signed provenance is the supported route: Google AI C2PA or current OpenAI C2PA with an explicit watermark action. -`verify-openai-synthid` is the official remote pixel check for OpenAI. -Research on a periodic lattice expert is in +Provider verifiers and provenance APIs are development-only oracles, not +user-facing runtime features. Research on a periodic lattice expert is in [synthid-detector-research.md](synthid-detector-research.md). For important outputs: @@ -192,7 +192,7 @@ are tracked in `data/evaluations/video-synthid-oracle.csv`. `remove_video_invisible` regenerates the video stream and copies the source audio verbatim: the extracted audio bitstream of input and output has an identical sha256, measured on two carriers. It also strips every metadata marker, so the -output can report clean from `get_ai_metadata` and `identify_video` while carrying +output can have no locally supported metadata or visible signal while carrying an untouched generated audio track. Google's verifier scores audio and visual tracks separately, and a track this path never modifies is a track it cannot have cleaned. @@ -202,6 +202,11 @@ been established -- that needs a provider verdict, which has not been obtained. Treat a clean local report on a clip with generated audio as unproven, not as a guarantee, and check the audio separately when it matters. +The public video results make that boundary explicit without decoding audio: +`visual_invisible_action` records whether video pixels were regenerated, while +the nested audio status reports `copied_if_present` and `unverified`. CLI output +uses the same terms. These fields add no media pass or model inference. + The shipped engine streams sampled frames in bounded batches, computes its fidelity metrics incrementally, and pipes regenerated pixels directly to ffmpeg. Its frame and latent memory is therefore bounded by `--batch-size` diff --git a/docs/module-internals.md b/docs/module-internals.md index 44cf86e..9f0d2df 100644 --- a/docs/module-internals.md +++ b/docs/module-internals.md @@ -188,6 +188,15 @@ successful output set has no silent holes. An invisible batch loads one VAE runtime and reuses it across every compatible file; a failed model load is reported per file without retrying the same multi-GB initialization. +Cleanup results separate an action from a watermark verdict. The retained +`invisible_removed` boolean means only that the visual regeneration stage ran; +`visual_invisible_action` exposes `not_run` or `regenerated` without implying a +fresh oracle result. `VideoAudioStatus` records `copied_if_present` and +`unverified` for metadata, visible, invisible, all, and batch outputs. It is +static result metadata: no extra probe, decode, model, or network call runs. A +visible no-op or failed batch item with no published output reports +`stream_action=not_written` instead. + Native MP4/MOV TC260 labels follow TC260-PG-20257A: `moov.udta.meta.keys` maps an `AIGC` key to a raw JSON value in `ilst`. [`_internal/isobmff.py`](../src/remove_ai_watermarks/_internal/isobmff.py) walks those @@ -277,7 +286,7 @@ current-frame fill. The same module supplies the motion-compensated metric used by the invisible-video sweep. [`video_invisible.py`](../src/remove_ai_watermarks/video_invisible.py) -implements the oracle-certified video SynthID removal engine. It samples frames +implements the calibrated video-pixel SynthID-removal engine. It samples frames uniformly, resizes to a VAE-aligned geometry, encodes each frame to latent space, applies one seeded spatial-noise field across the entire sequence, and decodes fresh pixels. Reusing a single noise field avoids independent @@ -624,7 +633,7 @@ evidence and never turns a local negative into a clean verdict. The result envelope also names the signal family, provider scope, backend, whether metadata contributed to the verdict, whether pixels were preserved, and an explicit reason for unsupported or indeterminate results. These fields -are shared with the official OpenAI verifier's JSON boundary. +are shared with the development OpenAI oracle's JSON boundary. The production router selects `synthid-periodic-tile-large-v1` above 10 through 18 megapixels when both dimensions are at least 2,048 pixels. It evaluates all @@ -923,14 +932,12 @@ fallback in its 1-10 megapixel domain, and large-v1 above 10 megapixels. A 20-image real-corpus drift check was byte-identical after the earlier v2 integration. The calibration history and caveats are in the linked detector research plan. -### Official OpenAI SynthID verifier +### Development OpenAI SynthID oracle [`openai_provenance.py`](../src/remove_ai_watermarks/openai_provenance.py) -provides the explicit remote production backend exposed as -`verify-openai-synthid`. It is intentionally separate from `identify`, because -one invocation uploads a sanitized raster to OpenAI. The CLI requires -`--acknowledge-upload`, and the optional OpenAI SDK lives in the independent -`verify` extra. +provides a development-only remote oracle. It is absent from the installed CLI +and top-level Python API, and `identify` never imports it. Development +dependencies include the optional OpenAI SDK. The backend accepts only PNG, JPEG, and WebP. It computes a decoded RGBA pixel fingerprint, removes AI provenance metadata into a temporary file through @@ -949,13 +956,13 @@ the endpoint, temporary basename, media type, byte count, timeout, retry policy, duration, HTTP status, error code, and request id when available, but omit the source path, image bytes, credentials, and decoded-pixel fingerprint. -`OpenAIProvenanceError` preserves the status, API error code, request id, +The internal `OpenAIProvenanceError` preserves the status, API error code, request id, `Retry-After` value, and a transient-only `retryable` flag. The library does not automatically act on that flag: an explicit caller invocation is required for every additional upload. Transport and schema failures remain errors rather than becoming `not_detected` or a local detector result. -The result remains provider-scoped and positive-evidence-only. `not_detected` +The development result remains provider-scoped and positive-evidence-only. `not_detected` does not mean human-created, and the official endpoint's published prohibition on repeated reverse-engineering or evasion queries prevents using this backend as an adaptive training or removal oracle. diff --git a/docs/python-api.md b/docs/python-api.md index 40790bc..9627991 100644 --- a/docs/python-api.md +++ b/docs/python-api.md @@ -7,28 +7,11 @@ Dependency groups are identical for the CLI and Python API. The default install covers metadata extraction, normalization, verdict logic, and stripping. Array/pixel APIs use `pixels`; visible removal uses `visible`; DWT-DCT detection uses `detect`; pixel photo classification uses `classify`; invisible image removal uses `qwen-zimage` and an NVIDIA GPU; and -visible video processing uses `video`. Video SynthID removal is a separate VAE -path that still runs on CPU and combines `video` and `diffusion`. Add `heif` +visible video processing uses `video`. The video-pixel SynthID-removal profile is +a separate VAE path that still runs on CPU and combines `video` and `diffusion`. Add `heif` independently when path-based pixel APIs must decode HEIC, HEIF, or AVIF. See the complete [feature-extra matrix](installation.md#feature-extras). -## Verify OpenAI SynthID - -```python -import remove_ai_watermarks as raiw - -result = raiw.verify_openai_synthid("input.png", acknowledge_upload=True) -print(result.status) -``` - -Official OpenAI JSON results expose `signal_family`, -`provider_scope`, `backend`, `metadata_used_for_verdict`, and -`pixels_preserved`. Remote transport and response failures raise `OpenAIProvenanceError`. Its -`status_code`, `error_code`, `request_id`, `retry_after`, and `retryable` -attributes let a caller implement bounded backoff or a circuit breaker without -turning an API outage into a false `not_detected` result. One function call still -performs at most one upload. - ## Remove visible marks Install `remove-ai-watermarks[visible]` before using the visible-removal API. @@ -528,8 +511,11 @@ when neither signal is found. This gives callers one predictable output path. It does not run lossy invisible regeneration by default. `include_invisible=True` explicitly adds VAE regeneration for MP4, MOV, or M4V. -`VideoAllResult.invisible_removed` reports whether the oracle-certified SynthID -stage ran. +`VideoAllResult.invisible_removed` remains the compatibility boolean for whether +the visual regeneration stage ran; it is an action record, not a per-file +watermark verdict. `visual_invisible_action` exposes the same distinction as +`not_run` or `regenerated`. `audio.stream_action` is `copied_if_present`, while +`audio.watermark_status` remains `unverified` because no audio decoder runs. Process a top-level directory sequentially: @@ -545,7 +531,10 @@ Batch modes are `all`, `visible`, and `metadata`. Successful visible no-ops are copied byte-for-byte, keeping the output directory complete. Per-file failures are returned in `VideoBatchItem.error`; they do not discard successful outputs. The invisible stage is available only as an explicit opt-in in `all` mode and -reuses one loaded VAE runtime across the batch. +reuses one loaded VAE runtime across the batch. Each item carries the same +`audio` status, and `visual_invisible_action` distinguishes regeneration from a +run that left invisible video pixels untouched. A failed item whose output was +not published reports `audio.stream_action=not_written`. ## Inspect and strip video metadata @@ -566,7 +555,9 @@ output is `input_clean.mp4`, leaving the source untouched. An explicit output must use the same container extension as the source. The returned `VideoMetadataResult` records the source, output, metadata detected -before removal, and any markers remaining after the verified strip. MP4/MOV +before removal, and any markers remaining after the verified strip. Its `audio` +field records that source audio is copied if present and not watermark-verified. +MP4/MOV inspection recognizes the native TC260 `AIGC` entry in `moov.udta.meta.keys/ilst` and the QuickTime-form `meta` variants Doubao's iOS export writes; its removal preserves container size and encoded @@ -577,7 +568,7 @@ stream-copy remux. AVI inspection reads `LIST/INFO/AIGC`, and FLV inspection reads `script.onMetaData.AIGC`; both use the same verified ffmpeg stream-copy removal path. -## Remove video SynthID +## Apply the video-pixel SynthID-removal profile Install `remove-ai-watermarks[video,diffusion]` before using the video SynthID API. @@ -603,6 +594,9 @@ source metadata, and publishes atomically. The default output is The returned `VideoInvisibleResult` includes output geometry, frame rate, frame count, paired PSNR, and the motion-compensated temporal-residual ratio. Those fields measure fidelity and flicker only. They are not a SynthID detector. +`visual_invisible_action` is `regenerated`; the nested `audio` status says +`copied_if_present` and `unverified`. These are action and evidence records, not +an assertion that every modality is clean. The default `noise_std=0.15` is the current full-clip oracle floor; `0.10` remained detected on the public eight-second Veo calibration carrier. The default profile is oracle-certified. Google does not publish a local @@ -677,8 +671,10 @@ cuts and disjoint masks keep the independent current fill. Pass The returned `VideoVisibleResult` records the selected `mark`, the total, detected, and removed frame counts, plus any AI metadata that survived the -output encode. The function returns `output=None` and writes no file when no -stable mark is selected. Video pixels are transcoded through ffmpeg while the +output encode. Its `audio` field records `copied_if_present` and `unverified`. +The function returns `output=None` and writes no file when no +stable mark is selected; that result records `audio.stream_action=not_written`. +Video pixels are transcoded through ffmpeg while the complete source audio stream is copied. The encoder preserves supported 8-bit source chroma sampling, color tags, MP4/MOV track timescale, and relative variable-frame timestamps. It also retains a non-zero source start PTS and the diff --git a/docs/supported-signals.md b/docs/supported-signals.md index dba01c8..5a93038 100644 --- a/docs/supported-signals.md +++ b/docs/supported-signals.md @@ -167,13 +167,9 @@ under Google's all-media watermark policy, and current OpenAI C2PA carrying an explicit `c2pa.watermarked.*` action. Legacy OpenAI C2PA without that action does not assert SynthID. -The optional `verify-openai-synthid` command is a separate official remote -verifier for supported OpenAI watermarks. It strips AI provenance metadata from -a temporary PNG, JPEG, or WebP copy, proves that decoded RGBA pixels are -unchanged, and uses only the API's SynthID result. It is therefore independent -of C2PA for its decision, but it is not local: the sanitized raster is uploaded -to OpenAI after explicit acknowledgement. It is intentionally excluded from -`identify` and its negative result remains inconclusive. +Provider verifiers and provenance APIs are reserved for development-time +calibration. They are not exposed through the installed CLI, the top-level +Python API, or `identify`; production reports only locally available evidence. Microsoft Paint can name `com.microsoft.invismark.1` in a C2PA soft-binding assertion. Inspection reports both that exact algorithm and its signed `value`, diff --git a/docs/synthid-detector-removal-plan.md b/docs/synthid-detector-removal-plan.md index 8b29350..46ba1be 100644 --- a/docs/synthid-detector-removal-plan.md +++ b/docs/synthid-detector-removal-plan.md @@ -3339,10 +3339,13 @@ production watermark decoder: same-provider non-assertion rows crossed at a high rate, general-control specificity failed, and ordinary JPEG destroyed all but one positive. A post-hoc threshold above the temporal control maximum would retain 19 unique OpenAI rows, but that threshold has observed the holdout and -is not a new expert. The official OpenAI verifier remains the provider-wide -pixel route. +is not a new expert. The official OpenAI verifier supplied the provider-wide +development oracle. -### 2026-08-14: production OpenAI verifier boundary +### 2026-08-14: original OpenAI verifier boundary + +> Historical note: the public command and top-level API described below were +> retired on 2026-09-05. The backend is now a development-only oracle. OpenAI's official Content Provenance API now supplies the production-grade OpenAI pixel verdict that the local experiments could not justify. The runtime @@ -3467,8 +3470,8 @@ one. ### 2026-08-15: the same answer from the OpenAI verifier -The equivalent frozen batch went through the official OpenAI endpoint on -`verify-openai-synthid`, two OpenAI positives in a source and `replace-chroma` +The equivalent frozen batch went through the official OpenAI endpoint using +the then-public verifier, two OpenAI positives in a source and `replace-chroma` pair each, four calls, one pass. Every arm returned `detected` with metadata stripped, decoded pixels preserved, and the C2PA outcome ignored. The candidates sit at `54.13 dB` and `53.97 dB` with residual RMS near `0.50`, the same diff --git a/docs/synthid-detector-research.md b/docs/synthid-detector-research.md index f43f298..417d60f 100644 --- a/docs/synthid-detector-research.md +++ b/docs/synthid-detector-research.md @@ -40,9 +40,10 @@ rejects `addWatermark` (`Cannot find field`). Current Gemini API docs state that all generated images include a SynthID watermark. There is no encoder-off Google path. -What the product uses for the *watermark* is signed provenance and -`verify-openai-synthid`. The periodic-lattice expert is research-only under -`scripts/synthid_runtime/` and is not called from `identify` or the CLI. +What the runtime product uses for the *watermark* is signed provenance. The +official OpenAI verifier and the periodic-lattice expert are development-only; +the latter lives under `scripts/synthid_runtime/` and is not called from +`identify` or the CLI. Lineage measurements of that expert are in [SynthID source classifiers](synthid-classifiers.md). @@ -433,7 +434,7 @@ Battery 2026-08-23, no official oracle. Local numbers: `.local-eval/synthid/prc-oklab-attack-2026-08-15/wild-hypotheses-2026-08-23.json`. Prepared attack rasters wait in `wild-attacks-2026-08-23/` for a later verifier window. Jacobian / adaptive queries against -`verify-openai-synthid` stay out: the endpoint forbids reverse-engineering. +the OpenAI provenance oracle stay out: the endpoint forbids reverse-engineering. ### Tested locally, not a mark @@ -585,7 +586,7 @@ forward-temporal AUC 0.53 vs L1 `not_detected`, 0.95 vs COCO, 0.97 vs COCO after a 2 px crop. The student learns OpenAI-versus-photo and still does not see the oracle mark contrast. -## Product remainder for the watermark +## Runtime product remainder for the watermark -Signed provenance (`identify`) and `verify-openai-synthid` (remote, explicit -upload). A research lattice miss is not a clean SynthID negative. +Signed provenance (`identify`). The remote OpenAI oracle is development-only, +and a research lattice miss is not a clean SynthID negative. diff --git a/docs/synthid-removal-research.md b/docs/synthid-removal-research.md index df210fc..1ae2c6a 100644 --- a/docs/synthid-removal-research.md +++ b/docs/synthid-removal-research.md @@ -65,8 +65,9 @@ presentation gate, not residual damage. Those rungs are in | Add unmarked context (frame, pad, hstack, collage) | 2026-08-19 | Official `not_detected` is a presentation gate. White pad 40% linear stays `detected`; photo fill at the same geometry is `not_detected`. See detector identity/layout | | `cebeuq/Synthid-Bypass` diffusion reconstruction | 2026-08-13 | 12 public pairs. Lattice and Gemini Verify go quiet on the *after* images because the image was redrawn, not because a payload was deleted. Faces and text are a quality claim for regeneration, not a local wipe | -Oracle: `verify_openai_synthid` after AI-metadata strip. Seeds s1, s2, s3 -as in [detector research](synthid-detector-research.md). Raw files: +Development oracle: the internal OpenAI provenance helper after AI-metadata +strip. Seeds s1, s2, s3 as in +[detector research](synthid-detector-research.md). Raw files: `.local-eval/synthid/prc-oklab-attack-2026-08-15/`. ## Band that actually carries the mark @@ -225,7 +226,7 @@ regeneration is already in [synthid.md](synthid.md#23-removal-attacks-and-forens | Kassis and Hengartner, [arXiv:2405.08363](https://arxiv.org/abs/2405.08363) (UnMarker, IEEE S&P 2025) | No decoder feedback. Two adversarial spectral optimizations. Breaks even some semantic watermarks (best remaining detection 43%) | Not production SynthID | Spectral disruption without an oracle is the honest analog of our 16-32 scramble, except UnMarker is optimized and we used a one-octave phase shuffle. Goonatilake: UnMarker TPR 98.28% at 0.1% FPR as a *forensic* leftover | | Tallam et al., [arXiv:2505.08234](https://arxiv.org/abs/2505.08234) (SemanticRegen) | Partial, label-free regeneration of main objects | Tree-Ring, StegaStamp, StableSig, DWT/DCT. Not SynthID | Partial redraw. Our collage / photo-pad `not_detected` is a presentation gate, not this attack | | Cao et al., [arXiv:2608.10166](https://arxiv.org/abs/2608.10166) (MarkNull, USENIX Security 2026) | On-manifold latent decorrelation via a public diffusion proxy. Claims 100% on 20 Imagen-3 Gemini-verify images. PSNR 25.36 dB, SSIM 0.80 | Small Gemini-verify set | Independent evidence that a no-box latent reconstruction can confuse Gemini. Does not meet this project's 40 dB / 0.99 SSIM release gate. Still generation, not a pixel-only wipe | -| Goonatilake and Ateniese, [arXiv:2605.09203](https://arxiv.org/abs/2605.09203) | Six removers all leave a forensic residue a ResNet-50 sees at >98% TPR @ 1% FPR | Applies to UnMarker, Zhao's WatermarkAttacker, CtrlRegen+ | Defeating `verify-openai-synthid` is not deniability. This is the product remainder | +| Goonatilake and Ateniese, [arXiv:2605.09203](https://arxiv.org/abs/2605.09203) | Six removers all leave a forensic residue a ResNet-50 sees at >98% TPR @ 1% FPR | Applies to UnMarker, Zhao's WatermarkAttacker, CtrlRegen+ | Defeating a provider oracle is not deniability. This is the product remainder | | An et al., [arXiv:2401.08573](https://arxiv.org/abs/2401.08573) (WAVES, ICML 2024) | 26 attacks on StegaStamp, Stable Signature, Tree-Ring. Regeneration, not JPEG, is the attack that matters. StegaStamp TPR@1%FPR 1.00 to 0.01; Tree-Ring 0.99 to 0.12 | Open watermarks | Protocol. Our blur-sigma-7 and 16-32 scramble are closer to WAVES "distortion" than to regeneration | | Wen et al., [arXiv:2305.20030](https://arxiv.org/abs/2305.20030) (Tree-Ring) | In-generation Fourier pattern in initial noise | Not SynthID | Kill is DDIM inversion plus latent wipe, not a pixel scramble. Lin and Juarez [arXiv:2506.10502](https://arxiv.org/abs/2506.10502) (USENIX 2025) remove it from public knowledge | | Fernandez et al., Stable Signature, ICCV 2023 | Fine-tuned VAE decoder | Not SynthID | Regeneration that replaces the decoder is in-family. Our foreign VAE at 22.3 dB did not kill OpenAI SynthID, which is the expected mismatch | diff --git a/docs/synthid.md b/docs/synthid.md index 5e1239d..bec7e7e 100644 --- a/docs/synthid.md +++ b/docs/synthid.md @@ -592,8 +592,8 @@ The external variant SynthID-O is available "through partnerships" only. This package does not ship a local payload decoder. Research on a periodic lattice expert is in [synthid-detector-research.md](synthid-detector-research.md). A 2026-08-23 literature map (Gowal, AWPD/FSNet, PRC, reverse-SynthID) is -in that page's external-literature section. The product routes are signed -provenance and `verify-openai-synthid`. +in that page's external-literature section. The runtime product route is signed +provenance; provider verifiers remain development-only oracles. ### 3.2 How our tool detects the supported carrier @@ -603,17 +603,17 @@ expert is research-only under `scripts/synthid_runtime/` and [synthid-detector-research.md](synthid-detector-research.md). It is not a payload decoder and is not called from `identify` or the CLI. -The product routes for SynthID are signed provenance, documented in the next -sections, and `verify-openai-synthid`. +The runtime product route for SynthID is signed provenance, documented in the +next sections. Provider verifiers are used only during development. -### 3.3 Official OpenAI pixel verification +### 3.3 Development-only OpenAI pixel oracle -`remove-ai-watermarks verify-openai-synthid image.png ---acknowledge-upload` uses OpenAI's official Content Provenance API for the -OpenAI half of the production detector. This is not the weak local period-8 -research expert and it does not use C2PA as a proxy. +The internal `openai_provenance` module uses OpenAI's official Content +Provenance API for bounded development experiments. It is not the weak local +period-8 research expert, does not use C2PA as a proxy, and is not exposed by +the installed CLI or top-level Python API. -Before making one request, the command strips AI provenance metadata into a +Before making one request, the helper strips AI provenance metadata into a temporary PNG, JPEG, or WebP file and compares hashes of the decoded RGBA raster before and after. It refuses the upload if any AI marker survived, if the format or pixels changed, or if the sanitized file exceeds 50 MiB. It then reads @@ -633,17 +633,18 @@ metadata-stripped, pixel-identical OpenAI images at 1536 by 1024 and 1024 by An oracle-positive Google SynthID image and a COCO photograph, sanitized through the same path, both returned OpenAI `SynthID not detected` and no Content Credentials. This directly validates pixel-only and provider-specific behavior -for the production design. Four files are only a functional smoke test, not a +for the development oracle. Four files are only a functional smoke test, not a false-positive calibration, and the credentialed SDK endpoint itself was not called because no API key was available. -This backend is deliberately excluded from `identify`: verification uploads -the sanitized raster to OpenAI, the endpoint is not eligible for Zero Data -Retention, and explicit acknowledgement is required. It is suitable for -individual supported OpenAI provenance checks, not adaptive detector fitting or -removal search. The API documentation prohibits repeated queries for watermark -reverse engineering or evasion. As with every detector in this project, -`not_detected` is absence of recognized evidence, not proof of human authorship. +This helper is deliberately excluded from `identify`, the installed CLI, and +the public Python API: verification uploads the sanitized raster to OpenAI, the +endpoint is not eligible for Zero Data Retention, and explicit acknowledgement +is required. It is suitable only for bounded development checks over frozen +inputs, not adaptive detector fitting or removal search. The API documentation +prohibits repeated queries for watermark reverse engineering or evasion. As +with every detector in this project, `not_detected` is absence of recognized +evidence, not proof of human authorship. ### 3.4 How our tool recognizes SynthID from provenance diff --git a/pyproject.toml b/pyproject.toml index 068c08e..e1efe32 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -154,11 +154,6 @@ classify = [ "tokenizers>=0.22,<0.23", "huggingface-hub>=0.20.0", ] -# Official remote OpenAI SynthID verification. The command strips AI provenance -# metadata and proves pixel identity before upload; it is never called implicitly. -verify = [ - "openai>=2.52.0", -] # Universal region eraser backend -- big-LaMa via onnxruntime (Carve/LaMa-ONNX, # Apache-2.0). CPU, no torch. Model (~200 MB) is downloaded on first use and # cached by huggingface_hub; it is never bundled in this repo. The default cv2 @@ -181,6 +176,9 @@ migan = [ dev = [ "remove-ai-watermarks[video]", "remove-ai-watermarks[detect]", + # Development-only OpenAI Content Provenance oracle. It is not exposed by + # the installed CLI or the public Python API. + "openai>=2.52.0", "pytest>=8.0.0", "pytest-cov>=4.1.0", "pytest-xdist>=3.5.0", @@ -191,7 +189,7 @@ dev = [ ] # ``qwen-zimage`` already pulls ``diffusion``; naming both would suggest diffusion is # independently sufficient for a removal, which it is not. -all = ["remove-ai-watermarks[video,heif,detect,trustmark,text-restoration,migan,classify,verify]"] +all = ["remove-ai-watermarks[video,heif,detect,trustmark,text-restoration,migan,classify]"] [project.scripts] remove-ai-watermarks = "remove_ai_watermarks.cli:main" diff --git a/src/remove_ai_watermarks/__init__.py b/src/remove_ai_watermarks/__init__.py index 0fdc74f..850f5ce 100644 --- a/src/remove_ai_watermarks/__init__.py +++ b/src/remove_ai_watermarks/__init__.py @@ -12,9 +12,8 @@ High-level API (lazy, so ``import remove_ai_watermarks`` stays cheap):: raiw.remove_video_all("in.mp4", "out.mp4") # visible + verified metadata raiw.remove_video_batch("videos", "videos_clean") # complete per-file results raiw.remove_video_metadata("in.mp4", "out.mp4") # verified metadata strip - raiw.remove_video_invisible("in.mp4", "out.mp4") # oracle-certified SynthID removal + raiw.remove_video_invisible("in.mp4", "out.mp4") # calibrated video-pixel regeneration raiw.remove_video_visible("in.mp4", "out.mp4") # stable visible video-mark removal - raiw.verify_openai_synthid("in.png", acknowledge_upload=True) # remote For a provenance verdict use the ``identify`` submodule:: @@ -41,8 +40,6 @@ __all__ = [ "BatchSummary", "InvisibleOptions", "MetadataStripIncomplete", - "OpenAIProvenanceError", - "OpenAISynthIDDetection", "RemoveAllResult", "VisibleRemovalResult", "__version__", @@ -57,7 +54,6 @@ __all__ = [ "remove_video_visible", "remove_visible", "remove_visible_detailed", - "verify_openai_synthid", "visible_provenance", ] @@ -75,11 +71,6 @@ if TYPE_CHECKING: remove_visible_detailed, visible_provenance, ) - from remove_ai_watermarks.openai_provenance import ( - OpenAIProvenanceError, - OpenAISynthIDDetection, - verify_openai_synthid, - ) from remove_ai_watermarks.video import ( identify_video, inspect_video_metadata, @@ -122,8 +113,4 @@ def __getattr__(name: str) -> object: from remove_ai_watermarks import video return getattr(video, name) - if name in ("OpenAIProvenanceError", "OpenAISynthIDDetection", "verify_openai_synthid"): - from remove_ai_watermarks import openai_provenance - - return getattr(openai_provenance, name) raise AttributeError(f"module {__name__!r} has no attribute {name!r}") diff --git a/src/remove_ai_watermarks/cli.py b/src/remove_ai_watermarks/cli.py index 3e540ed..4f2ff7d 100644 --- a/src/remove_ai_watermarks/cli.py +++ b/src/remove_ai_watermarks/cli.py @@ -1245,6 +1245,11 @@ def cmd_video_metadata( _print_metadata_not_a_clean_verdict() +def _print_video_audio_unverified() -> None: + """Report the copied-audio boundary without claiming a watermark verdict.""" + console.print(" Audio watermark: UNVERIFIED; source audio copied unchanged if present") + + @cmd_video.command("invisible") @click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path)) @_video_output_option @@ -1259,7 +1264,7 @@ def cmd_video_invisible( seed: int, device: str, ) -> None: - """Remove video SynthID with the oracle-certified VAE profile.""" + """Apply the calibrated SynthID-removal profile to video pixels.""" from remove_ai_watermarks.video import remove_video_invisible _banner() @@ -1282,9 +1287,11 @@ def cmd_video_invisible( console.print(f" FAILED: {len(result.remaining_metadata)} AI metadata marker(s) survived in {result.output}") raise SystemExit(1) console.print( - f" SynthID removal complete: {result.width}x{result.height}, " + " Video pixels: regenerated with the calibrated SynthID-removal profile " + f"(not a per-file verification): {result.width}x{result.height}, " f"{result.total_frames} frames at {result.fps:.4g} fps -> {result.output}" ) + _print_video_audio_unverified() @cmd_video.command("visible") @@ -1327,6 +1334,7 @@ def cmd_video_visible( f" Removed {result.mark} watermark from " f"{result.removed_frames}/{result.total_frames} frames -> {result.output}" ) + _print_video_audio_unverified() @cmd_video.command("all") @@ -1336,7 +1344,7 @@ def cmd_video_visible( @click.option( "--invisible/--no-invisible", default=False, - help="Opt into oracle-certified lossy video SynthID removal.", + help="Opt into calibrated lossy regeneration of the video pixels.", ) @_video_invisible_options def cmd_video_all( @@ -1390,7 +1398,10 @@ def cmd_video_all( ) console.print(f" AI metadata: stripped -> {result.output}") if result.invisible_removed: - console.print(" SynthID: removed with the oracle-certified VAE profile") + console.print( + " Video pixels: regenerated with the calibrated SynthID-removal profile (not a per-file verification)" + ) + _print_video_audio_unverified() @cmd_video.command("batch") @@ -1413,7 +1424,7 @@ def cmd_video_all( @click.option( "--invisible/--no-invisible", default=False, - help="Opt into oracle-certified lossy SynthID removal in all mode.", + help="Opt into calibrated lossy video-pixel regeneration in all mode.", ) @_video_invisible_options def cmd_video_batch( @@ -1469,57 +1480,13 @@ def cmd_video_batch( f" Batch complete: {result.processed} processed, {result.failed} failed -> {result.output_directory}" ) if result.invisible_removed: - console.print(f" SynthID: removed from {result.invisible_removed} file(s)") + console.print(f" Video pixel regeneration: applied to {result.invisible_removed} file(s)") + if result.processed: + _print_video_audio_unverified() if result.failed: raise SystemExit(1) -# ── Official OpenAI SynthID verification ── -@main.command("verify-openai-synthid") -@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path)) -@click.option( - "--acknowledge-upload", - is_flag=True, - help="Confirm upload of a pixel-identical, AI-metadata-stripped copy to OpenAI.", -) -@click.option("--json", "as_json", is_flag=True, help="Emit the verifier result as JSON.") -def cmd_verify_openai_synthid(source: Path, acknowledge_upload: bool, as_json: bool) -> None: - """Use OpenAI's official verifier on pixels, independently of C2PA. - - The command strips AI provenance metadata from a temporary copy, proves the - decoded pixels are unchanged, and uploads that copy to OpenAI. It reads only - the SynthID result. The source file is never modified. - """ - if not acknowledge_upload: - raise click.ClickException( - "this command uploads a temporary pixel-identical copy to OpenAI; pass --acknowledge-upload to continue" - ) - from remove_ai_watermarks.openai_provenance import verify_openai_synthid - - source = _validate_image(source) - try: - result = verify_openai_synthid(source, acknowledge_upload=True) - except (OSError, RuntimeError, ValueError) as exc: - raise click.ClickException(str(exc)) from exc - - if as_json: - click.echo(json.dumps(result.to_dict(), indent=2)) - return - - _banner() - console.print(f"\n OpenAI SynthID pixel watermark: {result.status}") - console.print(" Detector: official OpenAI Content Provenance API") - if result.model is not None: - console.print(f" Model: {result.model}") - if result.generated_at is not None: - console.print(f" Generated at: {result.generated_at}") - console.print( - " Input: AI provenance metadata was stripped and decoded pixels were preserved.\n" - " Scope: supported OpenAI SynthID only. A not_detected result is not proof\n" - " that the image is human-created or contains no other watermark." - ) - - # ── Provenance identification ── @main.command("identify") @click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path)) diff --git a/src/remove_ai_watermarks/openai_provenance.py b/src/remove_ai_watermarks/openai_provenance.py index 976ed5a..1d9482c 100644 --- a/src/remove_ai_watermarks/openai_provenance.py +++ b/src/remove_ai_watermarks/openai_provenance.py @@ -1,11 +1,11 @@ -"""Official OpenAI SynthID verification with metadata-independent input. +"""Development-only OpenAI SynthID oracle with metadata-independent input. The Content Provenance API returns C2PA and SynthID outcomes independently. This module removes AI provenance metadata before upload, proves that the decoded RGBA raster did not change, and then consumes only the SynthID result. -It is intentionally separate from :func:`identify`: calling it uploads one -sanitized raster to OpenAI and therefore always requires an explicit user -action. +It is intentionally absent from the public package API and CLI. Development +experiments call it explicitly, upload one sanitized raster to OpenAI, and +therefore always require an explicit acknowledgement. The OpenAI SDK is optional. Imports remain lazy so local and metadata-only paths do not acquire a network client dependency. @@ -29,7 +29,7 @@ log = logging.getLogger(__name__) OpenAISynthIDStatus = Literal["detected", "not_detected"] DETECTOR_ID = "openai-content-provenance-synthid-v1" -INSTALL_HINT = "install the verification extra: uv add 'remove-ai-watermarks[verify]'" +INSTALL_HINT = "install the development dependencies: uv sync --extra dev" MAX_UPLOAD_BYTES = 50 * 1024 * 1024 REQUEST_TIMEOUT_SECONDS = 120.0 # One acknowledgement authorizes one upload. The SDK otherwise retries some diff --git a/src/remove_ai_watermarks/video.py b/src/remove_ai_watermarks/video.py index 4d65586..cb4e90c 100644 --- a/src/remove_ai_watermarks/video.py +++ b/src/remove_ai_watermarks/video.py @@ -8,7 +8,7 @@ The visible pixel path reuses the image package's shared fill backends. from __future__ import annotations -from dataclasses import dataclass +from dataclasses import dataclass, field from pathlib import Path from tempfile import TemporaryDirectory from typing import TYPE_CHECKING, Literal @@ -56,14 +56,23 @@ class VideoMetadataReport: markers: dict[str, str] +@dataclass(frozen=True) +class VideoAudioStatus: + """Audio disposition for a video output without an audio watermark check.""" + + stream_action: Literal["copied_if_present", "not_written"] = "copied_if_present" + watermark_status: Literal["unverified"] = "unverified" + + @dataclass(frozen=True) class VideoMetadataResult: - """Result of a verified video metadata-removal operation.""" + """Result of a verified metadata strip that preserves source audio.""" source: Path output: Path detected: dict[str, str] remaining: dict[str, str] + audio: VideoAudioStatus = field(default_factory=VideoAudioStatus, init=False) @dataclass(frozen=True) @@ -84,7 +93,7 @@ class VideoProvenanceReport: @dataclass(frozen=True) class VideoVisibleResult: - """Result of visible AI-watermark removal from a video.""" + """Result of visible AI-watermark removal with unverified copied audio.""" source: Path output: Path | None @@ -93,17 +102,28 @@ class VideoVisibleResult: detected_frames: int removed_frames: int remaining_metadata: dict[str, str] + audio: VideoAudioStatus = field(default_factory=VideoAudioStatus, init=False) + + def __post_init__(self) -> None: + if self.output is None: + object.__setattr__(self, "audio", VideoAudioStatus(stream_action="not_written")) @dataclass(frozen=True) class VideoInvisibleResult: - """Result of removing video SynthID through the oracle-certified VAE profile.""" + """Result of applying the calibrated VAE profile to video pixels only.""" source: Path output: Path noise_std: float metrics: RegenerationMetrics remaining_metadata: dict[str, str] + audio: VideoAudioStatus = field(default_factory=VideoAudioStatus, init=False) + + @property + def visual_invisible_action(self) -> Literal["regenerated"]: + """Describe the pixel action without claiming a per-file watermark verdict.""" + return "regenerated" @property def total_frames(self) -> int: @@ -132,7 +152,7 @@ class VideoInvisibleResult: @dataclass(frozen=True) class VideoAllResult: - """Result of the complete video-cleaning pipeline.""" + """Result of visible, metadata, and optional video-pixel processing.""" source: Path output: Path @@ -143,11 +163,17 @@ class VideoAllResult: detected_metadata: dict[str, str] remaining_metadata: dict[str, str] invisible_removed: bool + audio: VideoAudioStatus = field(default_factory=VideoAudioStatus, init=False) + + @property + def visual_invisible_action(self) -> Literal["not_run", "regenerated"]: + """Describe whether the opt-in visual regeneration stage ran.""" + return "regenerated" if self.invisible_removed else "not_run" @dataclass(frozen=True) class VideoBatchItem: - """Outcome for one source in a video batch.""" + """Outcome for one source in a video batch, including the audio boundary.""" source: Path output: Path | None @@ -156,6 +182,16 @@ class VideoBatchItem: visible_mark: str | None invisible_removed: bool error: str | None = None + audio: VideoAudioStatus = field(default_factory=VideoAudioStatus, init=False) + + def __post_init__(self) -> None: + if self.output is None: + object.__setattr__(self, "audio", VideoAudioStatus(stream_action="not_written")) + + @property + def visual_invisible_action(self) -> Literal["not_run", "regenerated"]: + """Describe whether visual regeneration ran for this item.""" + return "regenerated" if self.invisible_removed else "not_run" @dataclass(frozen=True) @@ -507,7 +543,7 @@ def remove_video_all( device: str = "auto", _invisible_runtime: VideoVaeRuntime | None = None, ) -> VideoAllResult: - """Run the complete video cleaning pipeline. + """Run visible, metadata, and optional video-pixel processing. The default path removes a stable visible provider mark when present and always strips verified AI metadata. It writes a same-container passthrough @@ -760,11 +796,12 @@ def remove_video_invisible( device: str = "auto", _runtime: VideoVaeRuntime | None = None, ) -> VideoInvisibleResult: - """Remove video SynthID through the oracle-certified VAE profile. + """Regenerate video pixels through the calibrated SynthID-removal profile. The function also strips source metadata during the transcode. The default - profile is provider-oracle certified; important outputs may still be - rechecked with Google's verifier when the caller needs a per-file verdict. + profile was calibrated against a provider oracle, but this call performs no + per-file verification. Source audio is copied if present and its watermark + status remains unverified. """ from remove_ai_watermarks.metadata import get_ai_metadata diff --git a/src/remove_ai_watermarks/video_synthid.py b/src/remove_ai_watermarks/video_synthid.py index 81a4ae3..a34b647 100644 --- a/src/remove_ai_watermarks/video_synthid.py +++ b/src/remove_ai_watermarks/video_synthid.py @@ -1,4 +1,4 @@ -"""Shared configuration for oracle-certified video SynthID removal.""" +"""Shared configuration for the calibrated video-pixel SynthID-removal profile.""" DEFAULT_VIDEO_SYNTHID_VAE = "stabilityai/sd-vae-ft-mse" # The certified profile is a perturbation-to-signal ratio, so it is pinned against diff --git a/tests/test_api.py b/tests/test_api.py index e051d2d..efdca58 100644 --- a/tests/test_api.py +++ b/tests/test_api.py @@ -18,16 +18,11 @@ CHATGPT = SAMPLES / "chatgpt-1.png" class TestTopLevelExports: def test_lazy_reexports_resolve(self): - from remove_ai_watermarks import openai_provenance - assert raiw.BatchItemResult is api.BatchItemResult assert raiw.remove_visible is api.remove_visible assert raiw.remove_visible_detailed is api.remove_visible_detailed assert raiw.VisibleRemovalResult is api.VisibleRemovalResult assert raiw.visible_provenance is api.visible_provenance - assert raiw.verify_openai_synthid is openai_provenance.verify_openai_synthid - assert raiw.OpenAIProvenanceError is openai_provenance.OpenAIProvenanceError - assert raiw.OpenAISynthIDDetection is openai_provenance.OpenAISynthIDDetection def test_unknown_attribute_raises(self): with pytest.raises(AttributeError): @@ -39,6 +34,14 @@ class TestTopLevelExports: with pytest.raises(AttributeError): _ = raiw.SynthIDDetection + @pytest.mark.parametrize( + "name", + ["verify_openai_synthid", "OpenAIProvenanceError", "OpenAISynthIDDetection"], + ) + def test_development_openai_oracle_is_not_a_package_export(self, name: str): + with pytest.raises(AttributeError): + getattr(raiw, name) + def test_bare_import_is_light(self): # importing the package must not pull the heavy cv2/torch stack (PEP 562 lazy). # Checked in a FRESH interpreter -- another test in this process may already diff --git a/tests/test_cli.py b/tests/test_cli.py index 31465cb..7897e7d 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -848,80 +848,12 @@ class TestDetectSynthIDCommandRemoved: assert "No such command" in result.output -class TestVerifyOpenAISynthIDCommand: - def test_help_names_upload_and_pixel_independence(self, runner): +class TestVerifyOpenAISynthIDCommandRemoved: + def test_command_is_not_registered(self, runner): result = runner.invoke(main, ["verify-openai-synthid", "--help"]) - assert result.exit_code == 0 - assert "--acknowledge-upload" in result.output - assert "independently of C2PA" in result.output - - def test_upload_requires_explicit_acknowledgement(self, runner, tmp_clean_png, monkeypatch): - from remove_ai_watermarks import openai_provenance - - called = False - - def verify(_source, *, acknowledge_upload): - nonlocal called - assert acknowledge_upload is True - called = True - - monkeypatch.setattr(openai_provenance, "verify_openai_synthid", verify) - - result = runner.invoke(main, ["verify-openai-synthid", str(tmp_clean_png)]) - assert result.exit_code != 0 - assert "pass --acknowledge-upload" in result.output - assert called is False - - def test_json_result_is_machine_readable(self, runner, tmp_clean_png, monkeypatch): - from remove_ai_watermarks import openai_provenance - - expected = openai_provenance.OpenAISynthIDDetection( - status="not_detected", - model=None, - generated_at=None, - api_created_at=1_778_000_000, - ) - monkeypatch.setattr( - openai_provenance, - "verify_openai_synthid", - lambda _source, *, acknowledge_upload: expected if acknowledge_upload else None, - ) - - result = runner.invoke( - main, - ["verify-openai-synthid", str(tmp_clean_png), "--acknowledge-upload", "--json"], - ) - - assert result.exit_code == 0, result.output - payload = json.loads(result.output) - assert payload == expected.to_dict() - assert "c2pa" not in payload - - def test_text_result_preserves_negative_scope(self, runner, tmp_clean_png, monkeypatch): - from remove_ai_watermarks import openai_provenance - - expected = openai_provenance.OpenAISynthIDDetection( - status="not_detected", - model=None, - generated_at=None, - api_created_at=None, - ) - monkeypatch.setattr( - openai_provenance, - "verify_openai_synthid", - lambda _source, *, acknowledge_upload: expected if acknowledge_upload else None, - ) - - result = runner.invoke( - main, - ["verify-openai-synthid", str(tmp_clean_png), "--acknowledge-upload"], - ) - - assert result.exit_code == 0, result.output - assert "AI provenance metadata was stripped" in result.output - assert "not proof" in result.output + assert "No such command" in result.output class TestBatchCommand: diff --git a/tests/test_openai_provenance.py b/tests/test_openai_provenance.py index 77f9ee6..793d56b 100644 --- a/tests/test_openai_provenance.py +++ b/tests/test_openai_provenance.py @@ -277,7 +277,7 @@ def test_missing_optional_sdk_has_install_hint( ) -> None: monkeypatch.setattr(provenance, "is_available", lambda: False) - with pytest.raises(RuntimeError, match=r"remove-ai-watermarks\[verify\]"): + with pytest.raises(RuntimeError, match=r"uv sync --extra dev"): _verify(tmp_clean_png) diff --git a/tests/test_packaging.py b/tests/test_packaging.py index d8ab405..03655f6 100644 --- a/tests/test_packaging.py +++ b/tests/test_packaging.py @@ -61,7 +61,11 @@ def test_video_extra_owns_timestamp_dependency(): def test_file_format_and_detector_dependencies_are_independent(): assert "pillow-heif" in _requirement_names("heif") assert "pywavelets" in _requirement_names("detect") - assert "openai" in _requirement_names("verify") + + +def test_openai_sdk_is_development_only(): + assert "openai" in _requirement_names("dev") + assert "openai" not in _requirement_names("all") def test_trustmark_is_limited_to_its_numpy_compatible_python_range(): @@ -72,8 +76,8 @@ def test_trustmark_is_limited_to_its_numpy_compatible_python_range(): def test_extras_use_capability_names_without_legacy_aliases(): extras = set(metadata("remove-ai-watermarks").get_all("Provides-Extra") or []) - assert {"pixels", "heif", "visible", "video", "detect", "diffusion", "classify", "verify"} <= extras - assert {"gpu", "remove", "detect-pywavelets"}.isdisjoint(extras) + assert {"pixels", "heif", "visible", "video", "detect", "diffusion", "classify"} <= extras + assert {"gpu", "remove", "detect-pywavelets", "verify"}.isdisjoint(extras) def test_classify_extra_owns_the_photo_heads() -> None: diff --git a/tests/test_video.py b/tests/test_video.py index 0a7d409..b373b51 100644 --- a/tests/test_video.py +++ b/tests/test_video.py @@ -597,6 +597,8 @@ class TestVideoMetadataApi: assert result.output == output assert result.detected assert result.remaining == {} + assert result.audio.stream_action == "copied_if_present" + assert result.audio.watermark_status == "unverified" assert _VIDEO_PAYLOAD in output.read_bytes() assert C2PA_UUID not in output.read_bytes() @@ -1025,6 +1027,9 @@ class TestVideoAllApi: assert result.detected_metadata assert result.remaining_metadata == {} assert result.invisible_removed is False + assert result.visual_invisible_action == "not_run" + assert result.audio.stream_action == "copied_if_present" + assert result.audio.watermark_status == "unverified" assert C2PA_UUID not in output.read_bytes() assert _VIDEO_PAYLOAD in output.read_bytes() @@ -1116,6 +1121,9 @@ class TestVideoAllApi: result = remove_video_all(source, output, include_invisible=True) assert result.invisible_removed is True + assert result.visual_invisible_action == "regenerated" + assert result.audio.stream_action == "copied_if_present" + assert result.audio.watermark_status == "unverified" assert output.exists() assert len(intermediate_sources) == 1 assert not intermediate_sources[0].exists() @@ -1145,7 +1153,8 @@ class TestVideoAllCli: assert result.exit_code == 0, result.output assert "--invisible" in result.output - assert "oracle-certified" in result.output + assert "calibrated lossy regeneration" in result.output + assert "video pixels" in result.output def test_reports_locally_verified_result(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch): from remove_ai_watermarks import video @@ -1173,9 +1182,9 @@ class TestVideoAllCli: assert result.exit_code == 0, result.output assert "removed sora from 12/12 frames" in result.output - assert "UNVERIFIED" not in result.output + assert "Audio watermark: UNVERIFIED" in result.output - def test_reports_oracle_certified_invisible_removal( + def test_reports_video_regeneration_and_unverified_audio( self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch, @@ -1204,8 +1213,9 @@ class TestVideoAllCli: result = CliRunner().invoke(main, ["video", "all", str(source), "--invisible"]) assert result.exit_code == 0, result.output - assert "UNVERIFIED" not in result.output - assert "oracle-certified VAE profile" in result.output + assert "Video pixels: regenerated with the calibrated SynthID-removal profile" in result.output + assert "not a per-file verification" in result.output + assert "Audio watermark: UNVERIFIED" in result.output class TestVideoBatchApi: @@ -1248,7 +1258,9 @@ class TestVideoBatchApi: assert (result.output_directory / clean.name).read_bytes() == clean.read_bytes() assert result.items[0].source.name == "broken.mp4" assert result.items[0].error == "invalid container" + assert result.items[0].audio.stream_action == "not_written" assert result.items[1].changed is False + assert result.items[1].audio.stream_action == "copied_if_present" def test_metadata_mode_processes_every_supported_video(self, tmp_path: Path): from remove_ai_watermarks.video import remove_video_batch @@ -1399,6 +1411,44 @@ class TestVideoBatchCli: assert "FAILED broken.mp4: invalid container" in result.output assert "1 failed" in result.output + def test_reports_visual_regeneration_without_claiming_audio_removal( + self, + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, + ): + from remove_ai_watermarks import video + from remove_ai_watermarks.video import VideoBatchItem, VideoBatchResult + + directory = tmp_path / "videos" + directory.mkdir() + output = tmp_path / "clean" + source = directory / "source.mp4" + monkeypatch.setattr( + video, + "remove_video_batch", + lambda *_args, **_kwargs: VideoBatchResult( + directory=directory, + output_directory=output, + items=( + VideoBatchItem( + source=source, + output=output / source.name, + mode="all", + changed=True, + visible_mark=None, + invisible_removed=True, + ), + ), + ), + ) + + result = CliRunner().invoke(main, ["video", "batch", str(directory), "--invisible"]) + + assert result.exit_code == 0, result.output + assert "Video pixel regeneration: applied to 1 file(s)" in result.output + assert "SynthID: removed" not in result.output + assert "Audio watermark: UNVERIFIED" in result.output + class TestVideoInvisibleApi: def test_removes_synthid_and_strips_metadata( @@ -1423,6 +1473,9 @@ class TestVideoInvisibleApi: assert result.output == output assert result.total_frames == 24 assert result.remaining_metadata == {} + assert result.visual_invisible_action == "regenerated" + assert result.audio.stream_action == "copied_if_present" + assert result.audio.watermark_status == "unverified" def test_default_output_is_named_as_clean( self, @@ -1461,13 +1514,13 @@ class TestVideoInvisibleApi: class TestVideoInvisibleCli: - def test_help_describes_oracle_certification(self): + def test_help_describes_calibrated_video_pixel_action(self): runner = CliRunner() result = runner.invoke(main, ["video", "invisible", "--help"]) assert result.exit_code == 0, result.output - assert "oracle-certified" in result.output + assert "calibrated SynthID-removal profile to video pixels" in result.output def test_reports_completed_removal( self, @@ -1495,8 +1548,10 @@ class TestVideoInvisibleCli: result = runner.invoke(main, ["video", "invisible", str(source), "-o", str(output)]) assert result.exit_code == 0, result.output - assert "SynthID removal complete" in result.output - assert "UNVERIFIED" not in result.output + assert "SynthID removal complete" not in result.output + assert "Video pixels: regenerated with the calibrated SynthID-removal profile" in result.output + assert "not a per-file verification" in result.output + assert "Audio watermark: UNVERIFIED" in result.output class TestSoraFrameLocalization: @@ -2674,6 +2729,8 @@ class TestVideoVisibleApi: assert result.detected_frames == 5 assert result.removed_frames == 5 assert result.remaining_metadata == {} + assert result.audio.stream_action == "copied_if_present" + assert result.audio.watermark_status == "unverified" def test_no_stable_mark_writes_no_output(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch): from remove_ai_watermarks import video_visible @@ -2702,6 +2759,8 @@ class TestVideoVisibleApi: assert result.output is None assert result.removed_frames == 0 + assert result.audio.stream_action == "not_written" + assert result.audio.watermark_status == "unverified" assert not output.exists() def test_dispatches_veo_detector_and_uses_tighter_mask( @@ -2840,3 +2899,4 @@ class TestVideoVisibleCli: assert result.exit_code == 0, result.output assert "10/12 frames" in result.output + assert "Audio watermark: UNVERIFIED" in result.output diff --git a/uv.lock b/uv.lock index e679e5f..efd5bb0 100644 --- a/uv.lock +++ b/uv.lock @@ -3230,7 +3230,6 @@ all = [ { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.13'" }, { name = "numpy", version = "2.3.5", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.13'" }, { name = "onnxruntime" }, - { name = "openai" }, { name = "opencv-python-headless" }, { name = "pillow-heif" }, { name = "pywavelets" }, @@ -3262,6 +3261,7 @@ dev = [ { name = "invisible-watermark" }, { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.13'" }, { name = "numpy", version = "2.3.5", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.13'" }, + { name = "openai" }, { name = "opencv-python-headless" }, { name = "packaging" }, { name = "pyright" }, @@ -3342,9 +3342,6 @@ text-restoration = [ trustmark = [ { name = "trustmark", marker = "python_full_version < '3.13'" }, ] -verify = [ - { name = "openai" }, -] video = [ { name = "av" }, { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.13'" }, @@ -3374,7 +3371,7 @@ requires-dist = [ { name = "numpy", marker = "python_full_version < '3.13' and extra == 'pixels'", specifier = ">=1.24.0" }, { name = "onnxruntime", marker = "extra == 'lama'", specifier = ">=1.24.0" }, { name = "onnxruntime", marker = "extra == 'migan'", specifier = ">=1.24.0" }, - { name = "openai", marker = "extra == 'verify'", specifier = ">=2.52.0" }, + { name = "openai", marker = "extra == 'dev'", specifier = ">=2.52.0" }, { name = "opencv-python-headless", marker = "extra == 'pixels'", specifier = ">=4.8.0" }, { name = "packaging", marker = "extra == 'dev'", specifier = ">=24.0" }, { name = "paddleocr", marker = "extra == 'text-draft'", specifier = ">=3.3.3" }, @@ -3396,7 +3393,7 @@ requires-dist = [ { name = "remove-ai-watermarks", extras = ["pixels"], marker = "extra == 'visible'" }, { name = "remove-ai-watermarks", extras = ["qwen-zimage", "lama"], marker = "extra == 'text-restoration'" }, { name = "remove-ai-watermarks", extras = ["video"], marker = "extra == 'dev'" }, - { name = "remove-ai-watermarks", extras = ["video", "heif", "detect", "trustmark", "text-restoration", "migan", "classify", "verify"], marker = "extra == 'all'" }, + { name = "remove-ai-watermarks", extras = ["video", "heif", "detect", "trustmark", "text-restoration", "migan", "classify"], marker = "extra == 'all'" }, { name = "remove-ai-watermarks", extras = ["visible"], marker = "extra == 'lama'" }, { name = "remove-ai-watermarks", extras = ["visible"], marker = "extra == 'migan'" }, { name = "remove-ai-watermarks", extras = ["visible"], marker = "extra == 'video'" }, @@ -3412,7 +3409,7 @@ requires-dist = [ { name = "transformers", marker = "extra == 'diffusion'", specifier = ">=5,<6" }, { name = "trustmark", marker = "python_full_version < '3.13' and extra == 'trustmark'", specifier = ">=0.8.0" }, ] -provides-extras = ["pixels", "heif", "visible", "video", "detect", "diffusion", "qwen-zimage", "text-restoration", "text-draft", "trustmark", "classify", "verify", "lama", "migan", "dev", "all"] +provides-extras = ["pixels", "heif", "visible", "video", "detect", "diffusion", "qwen-zimage", "text-restoration", "text-draft", "trustmark", "classify", "lama", "migan", "dev", "all"] [[package]] name = "requests"