SdxlZImagePipeline subclasses QwenZImagePipeline and overrides only _run_global
and preload, so the face stage is inherited rather than copied and cannot drift
between the two profiles. A test asserts the shared methods are the same objects.
Four things are architecture-bound and swap with the model: the ControlNet, the
four-step distillation LoRA (SDXL-Lightning at its documented 1.0, not the
reference graph's 0.8, which belongs to a different LoRA), the sampler (Euler
trailing, no AuraFlow shift), and the latent grid at 8 px against Qwen's 16.
Strength is architecture-bound too, which is the easy mistake and cost two wrong
conclusions before it was caught. An SDXL global pass leaves SynthID at the
strength Qwen needs: through the Gemini app on a native 2816x1536 original, 0.154
is FOUND while 0.20, 0.25 and 0.30 are clean. The profile therefore takes a flat
vendor policy - OpenAI 0.15, Gemini 0.25, unknown following Gemini - rather than
resolution_adaptive_denoise, because flat values are what was measured and no
size dependence has been established for this stage.
requested_steps exists because the runtimes truncate differently: DiffSynth sets
sigma_start = denoising_strength and runs every requested step, while Diffusers
img2img truncates the step count, so four steps at 0.15 executes zero and returns
a bare VAE round-trip.
Also records both measured provider boundaries for the shipped qwen-zimage curve
- OpenAI detected at 0.06 and clean from 0.08, Gemini detected at 0.08 and clean
from 0.10 - together with the two low-resolution Gemini verdicts that explain why
the curve's sub-1 MP rungs are not under-driven despite looking short against a
boundary measured at 4.33 MP. The curve is left unchanged; nothing measured fails.
The profile is not deployed and not production-ready: every verdict so far comes
from one fixture and one seed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The parameter names invite a misreading. _run_faces regenerates the ENTIRE
expanded crop with Z-Image and only then cross-fades on a blurred SAM mask, so
the generation is conditioned on a fully noised neighbourhood and the pixels the
mask later discards were regenerated too.
Records the alternative that has never been tried here - passing the mask into
the sampler as a latent noise mask, so only masked pixels are denoised and the
edge transition happens inside the generation - because the face stage is the
largest measured quality contributor, worth 3.5 dB and 6.1 dB inside the face
boxes on the two fixtures that have one.
Also records that FACE_DENOISE_SCALE = 0.5 is coupled to this compositing choice
rather than independently calibrated: regenerating a whole crop and blending is
stronger than masked denoising, so changing the compositing without revisiting
the scale would move output strength by about a factor of two.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both qwen-zimage stages prompt with module constants, and at CFG 1.0 DiffSynth's
PipelineUnitRunner reuses the positive embedding for the negative side rather
than encoding it, so exactly one embedding per stage is ever computed. Persist it
and neither text encoder has to be loaded at all.
Measured on an H100 volume: this drops 15.45 GiB (Qwen2.5-VL) and 7.49 GiB
(Z-Image) of an 87.6 GiB per-request read, worth a median 11.76 s and 4.10 s of
load time paired within five containers. A nine-face fixture returned
sha256 c8567e11077de32a both with and without the cache, so the output is
byte-identical and the provider-oracle clearance is untouched.
The cache key carries the cache version, model id, pipeline output params and the
exact prompt, so a model bump or a prompt edit recomputes instead of reading a
stale embedding. The write is atomic because a torn file must never read back as
a hit, and a miss after the text encoder was already dropped raises rather than
calling a model that is not loaded.
_model_cache_dir now prefers HF_HOME: on a scale-to-zero runner that is the only
persistently mounted path, so anything below it is re-derived every request. The
YuNet download follows the same root.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Port of the same change made on the v0.20.1 line, reapplied here because the
package layout moved under _internal/ in the meantime.
The mandatory Qwen stack was configured to offload to disk unconditionally.
DiffSynth implements that by dropping the weights to the meta device and
re-reading every parameter through its DiskMap on the next onload, and the
pipeline moves between text encoder, transformer and VAE on every pass, so
each generation paid a full model reload. That is the right trade on a
consumer card, where it is what makes a 20B model runnable at all, and pure
waste on a card that can simply hold the stack.
Residency is now resolved from total VRAM, mirroring how the optional
Z-Image face stack is already gated. Above the floor the config passes no
"disk" value anywhere, which is what actually disables the behavior:
DiffSynth latches disk_offload once from offload_dtype, so pointing every
device at CUDA while leaving the sentinel would keep both the meta-drop and
the re-read.
Measured on an H100: a warm global pass went from 37.3s at 0.8 GiB resident
to 2.2s at 28.7 GiB, with both stacks resident peaking at 48.0 GiB of 79.2.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
New engines, each calibrated on its TC260 USCC cohort and validated by a
full-corpus sweep (42009 files):
- runninghub: top-left corner (new corner="tl"), faint mid-gray text via
the new raw-grayscale "gray" detection front-end, anchor-position gate
- baidu: text-run-only template (pill is a bright-blob magnet), load-bearing
Doubao+Qwen rival margins, corner-extended footprint for the white tag
- liblib: bottom-center (new corner="bc"), Arial silhouette (font is the
discriminative lever against latin UI text), logo-extended footprint
Qingyan parked (no clean-arm separation at any render/box), MiniMax/Hailuo
parked (1 visible frame, the xinghui rule); silhouettes kept as starting
points.
Calibrated on the 117-frame TC260-producer cohort (vendor_cohort_harvest +
vendor_mark_calibrate, both committed here): per-mark 2-rung ladder
(0.78, 1.27) for the two measured size modes, fitted locate box (the mark
sits ~0.025 of the short side off the edge; doubao's box clipped the first
glyph), measured template aspect 0.26, gate 0.45 (clean p99 0.301).
Strict-only (the sub-gate band is non-Qwen banners), no rival margin
(0 cross-fires on 400 doubao / 298 jimeng / 286 clean frames).
83/83 real marks detector-clean after cv2 fill.
TextMarkConfig gains a per-mark ladder field; the shipped 3-rung default
is unchanged for every other mark.
The Tier E adversarial sweep (new, scripts/robustness_suite.py) drove the real CLI
over truncated, corrupt, zero-byte, absurdly-shaped and bomb inputs, unicode and
RTL paths, hostile output directories and concurrent runs. It found two crashes;
the /simplify review then reproduced a third and worse one.
1. A FAILED WRITE CRASHED ON THE SIZE REPORT. image_io.imwrite is contractually
non-raising and returns False, but write_bgr_with_alpha discarded that bool and
returned None, so no caller could tell a failed write from a successful one.
Every write site then ran output.stat() to print the size, so a read-only
destination died with a bare FileNotFoundError pointing at the stat rather than
the write. The fix is deliberately NOT uniform: single-image commands exit via
the new cli._write_output_or_exit; api._write_visible_result RAISES so a library
caller gets an accurate error instead of a confusing FileNotFoundError from the
downstream metadata strip; and the batch sites raise but never SystemExit,
because the batch loop counts per-image exceptions and aborting would kill the
whole run.
2. BATCH LOST DATA SILENTLY. Into a read-only output directory it wrote ZERO files
for 2 inputs and exited 0 -- no traceback, no error, an empty output directory a
wrapping service would read as a completed run. The robustness harness could not
see this class at all, since it scored exit codes and traceback markers and this
failure has neither; it now asserts on the artifacts written.
3. A DIRECTORY PASSED AS THE IMAGE crashed the metadata scanner with
IsADirectoryError, because click.Path(exists=True) accepts directories. Fixed
with dir_okay=False on all six source arguments, so argument parsing refuses it.
Also adds Tier B4 (scripts/resource_ceilings.py): peak RSS per fill backend from
1 MP to 25 MP, one fresh process per cell. migan 603->775 MB and lama 4679->4779 MB
are flat in input size, confirming the crop-around-the-mask design and both
documented figures; cv2 is the only backend that grows (74->440 MB, 5.9x). The
harness's own no-op check originally allocated a full-frame temp before reading
peak RSS and inflated the numbers with input size -- it now compares only the mask
box, and the conclusion survived re-measurement.
And scripts/real_examples_e2e.py, which drives every command over real corpus
examples and checks the outcome rather than the exit code: 6/6 provenance classes
identified, 10/10 metadata strips re-scan clean, all three fill backends write,
diffusion on MPS writes genuinely changed images. It records samsung as a real
partial (the faintest mark, 0.431 -> 0.404 against a 0.40 gate on the weakest of
its 3 corpus positives) and treats the gated pill's refusal to act as correct.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The faint-mask fallback added for the tophat front-end thresholded the
max-normalized uint8 response at 0.5 -- which selects every non-zero pixel,
not "half the peak" as its comment claimed -- and filled ~120% of the corner
box on textured frames. Measured on 14 real faint-path frames (cv2 fill,
detector re-run after): the detector's own best-match box fills a 58.7%-median
corner box vs 120.9% for the threshold, both 100% detector-clean. Detection and
the mask now read one method, _tophat_best, whose score gates detection and
whose argmax box bounds the fill, so the two cannot drift by construction --
which is how the mismatch arose. The 0.5 constant is deleted.
Parity could not catch this (a mask that fills everything is trivially
detector-clean) and the regression test could not either: its flat fixture
gives every threshold the same box, so mutating the constant to 99.0 stayed
green. The fixture now carries texture and asserts the mask area is bounded,
not merely non-empty; it reproduces the corpus number (127% pre-fix).
Also lands the Tier B2 verification harnesses that found and bounded this:
- detector_response.py: response curves (detected AND maskable per cell); found
the size response is a comb, contrast is near-irrelevant, no unmaskable cells.
- ladder_headroom.py: measured that a denser scale ladder recovers 7.6% of
misses for a 2.52%->3.05% false-fire rise, and the one landscape rung that
helps is a geometry shift that helps and hurts equally (1.7:1) -- do not add.
- cjk_tail_probe.py: a generic shared-tail (AI生成) template does not separate
uncovered vendors from clean corners (0.407 vs clean p99 0.298).
Records the visible-parity re-run confirming the earlier front-end fix (doubao
91.8% -> 99.3%), and dedups the thrice-written stamp forward model into one
fill_quality.composite.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The SDXL removal pipelines (sdxl + controlnet) were built without
add_watermarker=False, so diffusers embedded its default open "Stable
Diffusion XL" DWT-DCT invisible watermark on every output whenever
invisible-watermark is installed (the detect extra). A watermark REMOVER was
therefore replacing one detectable AI watermark (SynthID) with another: the
cleaned output re-read as AI (identify -> "Open invisible watermark: Stable
Diffusion XL"), observed on the SynthID validation sample.
Both SDXL loaders now call a shared _disable_sdxl_watermarker helper (mirrors
_maybe_add_fp16_vae; the ControlNetModel sub-model and the Qwen loader never
call it, since only the pipeline accepts the kwarg). Verified end to end: the
affected outputs re-run clean (is_ai=None, no open watermark).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bug fixes (each with a regression test):
- metadata strip parity across every marker placement: IPTC digitalSourceType
in XMP, the Samsung post-EOI trailer, the China TC260 AIGC block in EXIF
UserComment, a bare AIGC block in a non-standard APP segment, and the ISOBMFF
EXIF path (AIGC + xAI) are all now stripped -- anything a scanner flags, the
strip reaches
- Samsung genAIType detected when its trailer sits past the 512 KB scan window
(file-tail read on large photos)
- crashes on edge inputs: Gemini detector on images with a short side < 16px,
footprint_mask on a zero-size ndarray, the humanizer on chromatic_shift >=
width, and the CLI on unreadable/corrupt/empty input (clean error, not a
traceback)
- WebP written losslessly (cv2 quality 101), not lossy at 100
- the IPTC digitalSourceType algorithmicMedia (procedural, not trained on
sampled data) is no longer flagged as AI-generated, so clean procedural
content is not scrubbed
- c2pa source-type: compositeWithTrainedAlgorithmicMedia is checked before the
bare algorithmicMedia token, so an AI-enhanced composite is not misclassified
Detection:
- integrity-clash coverage now normalizes ByteDance / Canva / ElevenLabs /
Black Forest Labs, so a transplanted manifest next to an independent
conflicting stamp is caught; the generic China TC260 AIGC label is attributed
to a co-present TC260 vendor, so a legit Doubao image (its own C2PA + TC260
label) does not clash (corpus-validated: 0 new clashes on 5000 carriers)
CLI:
- batch exits non-zero (with a warning) when any image errors or a GPU-missing
SynthID scrub is skipped, and copies the input through so the output dir stays
complete -- it used to always exit 0 and could silently drop files
Perf:
- GeminiEngine reused as a process-wide singleton with a precomputed template
ladder: -24% on the identify sparkle path, detection byte-identical
Internal: one shared _ai_exif_targets rule set feeds both EXIF scrubbers so
their coverage cannot drift; docs synced; maintain.sh hardened so the uv-secure
internal teardown crash no longer aborts the gate (still fails on a real finding).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
erase_migan fed the whole frame to the ONNX model, so peak RSS scaled with the
upload (~0.6 GB at 4 MP up to ~2.4 GB at 25 MP). Mirror erase_lama: crop a padded
region around the mask (pad = max(256, 2*bbox)), feed only that crop (at native
resolution -- MI-GAN accepts arbitrary dims, unlike LaMa's fixed 512 square), and
paste only masked pixels back. Peak RSS is now bounded by the mark size
(~0.6-0.9 GB), so a memory-tight host (a 1-2 GB web worker) can run MI-GAN on a
25 MP upload.
Fill quality is unchanged: verified by eye on real Gemini/Doubao marks plus a
ground-truth reconstruction sweep -- a tighter view if anything reduces the GAN's
hallucination of large background structure.
Extract the shared padded-crop-box math into _padded_crop_box (used by both
erase_lama and erase_migan).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two AI-provenance metadata types mined from the retained corpus that
identify previously read as no-signal:
- Dreamina (ByteDance's international Jimeng brand) signs C2PA as
"Bytedance Pte. Ltd." with a "Dreamina/x.y" claim generator and NO
digitalSourceType, so the generator name is the only AI signal. Add a
C2paAiVendor row with a new asserts_ai flag (identity-AI: presence
asserts AI without trainedAlgorithmicMedia) plus the derived
C2PA_IDENTITY_AI_ORGS view, folded into identify's c2pa_is_ai. Keyed on
the Dreamina generator token, not the "Bytedance Pte" issuer, so non-AI
CapCut edits signed by the same entity stay unattributed. 7/7 corpus
files now attribute to ByteDance.
- Tencent Cloud's TC260 AIGC variant uses a ServiceProvider/ServiceUser
schema (vs the producer-side ContentProducer schema), embedded in EXIF
ImageDescription; add those field names to _TC260_FIELDS so the generic
{"AIGC":{...}} gate accepts it. 11/11 corpus files now flagged.
Test-first: reproducing tests in test_identify.py / test_metadata.py.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The FP gate demotes a low-gradient match, but a real FAINT sparkle also has soft
edges, so metadata-stripped faint sparkles were dropped. Keep a low-grad match
that is a strong (conf >= 0.52), bright, near-WHITE-core sparkle: a real sparkle
core is white, a clean bright corner that shape-matches (sky/sun) is colored
(_core_saturation). Recovers ~14/20 stripped faint sparkles under the DEFAULT
strict/auto (no metadata, no flag) at ~1.25% clean false-fire (baseline 0.55%);
the ~0.51-scoring bright-background FPs stay demoted (below 0.52).
A learned classifier on the same features measured WORSE than the tuned gate
(tier-1: MLP 86.7% recall vs the gate's 90.8% at equal false-fire), so the
heuristic stays; a patch-CNN with richer features is roadmapped P2 with low
expected value -- the precision/recall wall is fundamental (deep-research +
tier-1 both confirm it).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Full-dataset validation of reverse-alpha (v0.12.1) vs the current localize->fill:
doubao/jimeng identical (100% coverage + clearance across all backends); gemini
strict coverage a few points below reverse-alpha (the FP tightening), every
missed mark recovered under assume_ai, clearance ~98% both, no outside-box
damage. Clearance is fill-independent (cv2/MI-GAN/LaMa all strip the mark shape);
the difference is visual fill quality on textured/structured backgrounds -- LaMa
best, MI-GAN can ghost/hallucinate, cv2 smears -- which motivates auto = LaMa >
MI-GAN > cv2. Added to module-internals, known-limitations, and the CLAUDE.md
compact list.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The auto backend now resolves best-first: LaMa (highest quality, recovers the
textured/structured backgrounds the classical fill smears) > MI-GAN > cv2. Both
learned backends share the same onnxruntime availability check, so auto cannot
tell them apart and always prefers the better one; a memory-tight deployment
that cannot afford LaMa's ~4.7 GB peak pins MI-GAN explicitly via
`--backend migan` / `backend="migan"` (the deployment's call, not the library's).
cv2 stays the no-deps floor and now emits a one-time quality warning when auto
falls back to it, since it smears texture/structure.
Motivated by a v0.12.1 reverse-alpha vs 0.14 localize->fill head-to-head:
reverse-alpha recovered structured backgrounds more cleanly than any inpaint;
LaMa closes most of that gap, MI-GAN can ghost/hallucinate, cv2 is weakest.
doubao/jimeng removal is identical between versions; gemini strict coverage is
4pp lower (all recovered via assume_ai) with cleaner clearance and no
outside-box damage.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Candidate carries only the fields the arbiter reads (key, label,
detected_strict, detected_relaxed, features); location/region/confidence were
vestigial from the removed best_auto_mark max-by-confidence path.
- resolve_backend returns preferred_inpaint_backend() directly (typed Literal)
instead of an identity ternary.
- colour/normalise/behaviour -> US spelling across code comments and docs.
No behavior change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve 10 code-review findings on the v0.14.0 localize->fill path, several
release-blocking:
- gemini: build the removal mask from the decision's provenance-aware region
instead of a strict internal re-detect. A relaxed/assume_ai sparkle was
re-demoted by the FP gate into a None mask and reported removed while left in
the image; this also drops the redundant double-detect.
- registry: report a mark removed only when a fill actually happened (remove()
returns a None region for an empty mask), so a no-op is never claimed.
- api/cli: add write_noop so the CLI `visible` no-mark path writes nothing and
cannot clobber a pre-existing -o file (was write-then-unlink -> data loss);
create output.parent; skip the same-file copy (SameFileError on in-place).
- cli: catch the missing migan/lama backend RuntimeError on the visible/all
paths (matches `erase`); route the single-mark relaxation through the shared
resolve_relax instead of an inline copy.
- metadata: keep_standard=False no longer takes the AI-only lossless JPEG
short-circuit (it left standard metadata); defer a malformed-marker JPEG to
the PIL fallback instead of reporting a partial strip as complete.
- invisible: register the HEIF opener before Image.open (HEIC --force) and
RGB-convert before the PNG temp (CMYK JPEG).
- pill: normalize via to_bgr so a 4-channel BGRA array cannot crash cvtColor.
Regression tests for each; docs synced (resolve_relax, write_noop,
best_auto_mark -> detect_marks).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Replace reverse-alpha removal with localize -> fill (template-free mask + one
shared cv2/MI-GAN/big-LaMa fill) for every mark; drops the colour-shift / dark-pit
failure modes, version-robust to a moved or re-rendered mark
- Separate perception/decision/action: engines report Candidates, a pure
decide(candidates, Context) arbiter owns all policy (sensitivity + provenance +
pill gate), remove_auto_marks orchestrates -- behavior-preserving (corpus 46/46/92)
- Three orthogonal knobs replace --method: --backend cv2|migan|lama,
--sensitivity auto|strict|assume-ai, provenance (auto from metadata)
- Add high-level api.remove_visible / visible_provenance (lazy top-level re-export);
visible --mark auto delegates to it so CLI and library share ONE path
- Read+write HEIC/AVIF on the pixel path via pillow-heif; imwrite preserves the input
format at max quality (JPEG q100/4:4:4); a no-op copies the original bytes verbatim
- Lossless byte-level JPEG metadata strip (no DCT re-encode); consolidate the two
remove_ai_metadata into one, delete legacy noai/cleaner + best_auto_mark
- Bump 0.13.0 -> 0.14.0
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Verified 0.13.0 pill removal on a 32k real-upload corpus. The metadata-OR-wordmark
gate was only ~1/3 precise: TC260 metadata confirms Jimeng-class provenance, not pill
presence, so the weak edge-NCC detector's false fires (textured ceilings/walls, where
inpaint visibly smears) were admitted whenever metadata was present.
Split into two arms (_keep_pill): the reliable bottom-right wordmark (~94% precise,
survives metadata stripping) removes the pill unrestricted; the metadata-only arm
removes it ONLY when the top-left footprint is flat enough for an invisible inpaint
(PillEngine.footprint_is_flat, median-Sobel <= _FLAT_TEXTURE_MAX). Keeps real
flat-scene pills and harmless flat false fires; leaves the damaging textured false
fires untouched. Corpus: 270 -> 118 removals, ~90 true preserved, damaging FP -> ~0.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add the Jimeng-basic top-left "AI生成" pill as a CAPTURE-LESS mark
(pill_engine.py): synthetic-silhouette edge-NCC detect + inpaint-only removal.
Gated in remove_auto_marks: kept only when Jimeng is confirmed (TC260 metadata
OR the bottom-right "★ 即梦AI" wordmark fired -- the wordmark keeps recall on
metadata-STRIPPED uploads) AND Doubao did not fire.
- Add an inpaint-fallback removal path + MI-GAN ONNX backend (migan extra, MIT,
~28 MB / ~1 GB peak -- droplet-friendly) alongside big-LaMa. New
--method auto|reverse-alpha|inpaint (shared across visible/all/batch) and
erase --backend migan; footprint_mask on each engine.
- auto is deterministic: reverse-alpha for capture marks (recovers exact pixels,
lighter -- measured cleaner than MI-GAN on structured backgrounds) and inpaint
only for the capture-less pill.
- --mark auto now removes EVERY detected mark in one pass (remove_auto_marks),
so a Jimeng-basic image's top-left pill AND bottom-right wordmark both clear.
- Bump 0.12.1 -> 0.13.0.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Bright-background photos/renders and a tiny app icon were flagged as
AI-generated by the visible detectors. Two failure modes:
- Gemini sparkle on a bright background (snow+sky photo, white product
render) scored ~0.51. The FP gate only demoted on a low core-ring
brightness margin, which a bright background makes high. Add a gradient
floor (_SPARKLE_FP_GRAD 0.55): a real sparkle is a crisp star (grad
~0.97-1.0), a smooth luminance blob that NCC-matches the diamond is not
(the two FPs measured grad 0.105 / 0.463). The OR is a strict superset
of the old margin-only demotion, so it cannot regress dark/mid (kept by
margin) or white-bg (kept by confidence) real sparkles.
- A 48x48 geometric icon matched the Doubao/Jimeng CJK silhouette at
0.41/0.47 NCC. Purely a small-size artifact (the same icon at >=256px
collapses to ~0.06-0.10). Guard text-mark detection below a 200px short
side (_MIN_DETECT_SHORT_SIDE); real marks ship on full-resolution
renders (smallest captured sample 1086px).
Corpus re-sweep flips only OpenAI content and already-cleaned outputs,
all sub-0.5, so no provenance verdict changes. Add synthetic regression
fixtures for both modes; docs/module-internals.md updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
