fix: validate Qwen hybrid runtime placement

This commit is contained in:
Joseph Magly
2026-08-27 01:12:20 -04:00
parent 477b14fc61
commit 827ba8e106
8 changed files with 273 additions and 14 deletions
+21
View File
@@ -86,6 +86,27 @@ uv pip check --python .venv/bin/python
.venv/bin/python scripts/run_conditional_gate.py bitsandbytes-runtime
```
### Qwen3.5/Qwen3.8 hybrid runtime
The Qwen3.5/Qwen3.8 Gated DeltaNet path requires both FLA and causal-conv1d.
Install the optional extra only after selecting the CUDA build of PyTorch:
```bash
CUDA_HOME=/usr/local/cuda uv sync --locked --extra dev --extra qwen-hybrid
.venv/bin/python -c 'import torch, fla, causal_conv1d; print(torch.__version__)'
```
`causal-conv1d` wheels are keyed to the Python, PyTorch, CUDA, platform, and C++
ABI combination. Prefer an exact upstream wheel when one exists. A source build
must use a `CUDA_HOME` whose major version matches `torch.version.cuda`; the
system-default `nvcc` may point at a different toolkit. OBLITERATUS fails before
weight allocation if either extension cannot be imported.
The complete Qwen hybrid text model must fit one CUDA device with 15% free-memory
headroom. Generic `device_map="auto"` layer sharding is deliberately disabled for
this architecture because the recurrent-state execution path has not been
validated across devices.
Use the virtual environment interpreter directly after replacing Torch. A subsequent
`uv run` or `uv sync` without the CUDA override may restore the portable CPU wheel
from the lock. The agentic developer installer automates this ordering and performs