mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-08-09 23:50:40 +02:00
Serve the static prompt embeddings from a persistent cache
Both qwen-zimage stages prompt with module constants, and at CFG 1.0 DiffSynth's PipelineUnitRunner reuses the positive embedding for the negative side rather than encoding it, so exactly one embedding per stage is ever computed. Persist it and neither text encoder has to be loaded at all. Measured on an H100 volume: this drops 15.45 GiB (Qwen2.5-VL) and 7.49 GiB (Z-Image) of an 87.6 GiB per-request read, worth a median 11.76 s and 4.10 s of load time paired within five containers. A nine-face fixture returned sha256 c8567e11077de32a both with and without the cache, so the output is byte-identical and the provider-oracle clearance is untouched. The cache key carries the cache version, model id, pipeline output params and the exact prompt, so a model bump or a prompt edit recomputes instead of reading a stale embedding. The write is atomic because a torn file must never read back as a hit, and a miss after the text encoder was already dropped raises rather than calling a model that is not loaded. _model_cache_dir now prefers HF_HOME: on a scale-to-zero runner that is the only persistently mounted path, so anything below it is re-derived every request. The YuNet download follows the same root. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
94730f77d3
commit
39523aba09
@@ -564,6 +564,39 @@ orchestration, YuNet integration, SAM selection, masks, sizing helpers, and pixe
|
||||
compositing are implemented for this runtime. Changing a calibrated model input
|
||||
requires the same provider-oracle and identity evaluation as a model change.
|
||||
|
||||
### Static prompt embeddings
|
||||
|
||||
Both stages prompt with module constants, and at CFG 1.0 DiffSynth's
|
||||
`PipelineUnitRunner` reuses the positive embedding for the negative side instead of
|
||||
encoding it. So exactly one embedding per stage is ever computed, from text that
|
||||
cannot vary at runtime, which makes it cacheable across containers rather than only
|
||||
within one pipeline.
|
||||
|
||||
`_cache_static_prompt_embeddings` therefore persists what the text encoder produced
|
||||
under `_model_cache_dir()/prompt-embeddings`, keyed by cache version, model id,
|
||||
pipeline output params, and the exact prompt string. Once that file exists,
|
||||
`_load_qwen` and `_load_zimage` drop the text-encoder `ModelConfig` from the model
|
||||
stack entirely and serve the stored tensors instead. Measured on an H100 volume in
|
||||
August 2026, that removes **15.45 GiB** (Qwen2.5-VL) and **7.49 GiB** (Z-Image) of a
|
||||
**87.6 GiB** per-request read, worth a median **11.76 s** and **4.10 s** of load time
|
||||
(paired within five containers). The output is **byte-identical** -- the stored
|
||||
tensors are the encoder's own -- so this needs no provider-oracle re-verification.
|
||||
|
||||
Three properties are load-bearing:
|
||||
|
||||
- **The key self-heals.** A model bump or a prompt edit changes the key, so the next
|
||||
container recomputes rather than reading a stale embedding. `_PROMPT_CACHE_VERSION`
|
||||
covers a change to the stored shape itself.
|
||||
- **The write is atomic.** A torn write must never be readable as a cache hit, so the
|
||||
payload lands in a temp file and is renamed into place.
|
||||
- **A miss after the encoder was dropped raises.** `require_cache` records that the
|
||||
stack was built without a text encoder on the strength of the file; falling back
|
||||
would call a model that is not loaded, which surfaces as an opaque crash.
|
||||
|
||||
`_model_cache_dir()` prefers `HF_HOME` for the same reason: on a scale-to-zero runner
|
||||
that is the only persistently mounted path, and anything below it is re-derived per
|
||||
request. The YuNet download follows the same root.
|
||||
|
||||
Regression coverage:
|
||||
|
||||
- [`test_qwen_zimage_pipeline.py`](../tests/test_qwen_zimage_pipeline.py)
|
||||
|
||||
Reference in New Issue
Block a user