mirror of
https://github.com/elder-plinius/OBLITERATUS.git
synced 2026-08-30 06:30:37 +02:00
fix: validate Qwen hybrid runtime placement
This commit is contained in:
@@ -86,6 +86,27 @@ uv pip check --python .venv/bin/python
|
||||
.venv/bin/python scripts/run_conditional_gate.py bitsandbytes-runtime
|
||||
```
|
||||
|
||||
### Qwen3.5/Qwen3.8 hybrid runtime
|
||||
|
||||
The Qwen3.5/Qwen3.8 Gated DeltaNet path requires both FLA and causal-conv1d.
|
||||
Install the optional extra only after selecting the CUDA build of PyTorch:
|
||||
|
||||
```bash
|
||||
CUDA_HOME=/usr/local/cuda uv sync --locked --extra dev --extra qwen-hybrid
|
||||
.venv/bin/python -c 'import torch, fla, causal_conv1d; print(torch.__version__)'
|
||||
```
|
||||
|
||||
`causal-conv1d` wheels are keyed to the Python, PyTorch, CUDA, platform, and C++
|
||||
ABI combination. Prefer an exact upstream wheel when one exists. A source build
|
||||
must use a `CUDA_HOME` whose major version matches `torch.version.cuda`; the
|
||||
system-default `nvcc` may point at a different toolkit. OBLITERATUS fails before
|
||||
weight allocation if either extension cannot be imported.
|
||||
|
||||
The complete Qwen hybrid text model must fit one CUDA device with 15% free-memory
|
||||
headroom. Generic `device_map="auto"` layer sharding is deliberately disabled for
|
||||
this architecture because the recurrent-state execution path has not been
|
||||
validated across devices.
|
||||
|
||||
Use the virtual environment interpreter directly after replacing Torch. A subsequent
|
||||
`uv run` or `uv sync` without the CUDA override may restore the portable CPU wheel
|
||||
from the lock. The agentic developer installer automates this ordering and performs
|
||||
|
||||
Reference in New Issue
Block a user