mirror of
https://github.com/elder-plinius/OBLITERATUS.git
synced 2026-09-22 09:21:04 +02:00
Pliny the YoungerandClaude Fable 5.1
89b63ce216
transformers 5 stores routed experts as fused 3D parameters (experts.gate_up_proj / experts.down_proj) for Mixtral, Qwen3-MoE, DeepSeek-V3, GLM-4 MoE, Llama 4, gpt-oss and OLMoE. When such a layer is CPU- or disk-offloaded by Accelerate, EXCISE failed closed with "offloaded fused expert tensors are not yet a supported surgery layout", so every frontier MoE checkpoint large enough to need offload could not be abliterated at all. Route the three fused projections and the fused bias projection through the existing LogicalParameterTransaction adapter: a meta-resident fused parameter is resolved to a private copy of its authoritative backing value, projected per expert, and committed atomically; the live parameter stays on meta and quantized or unknown backing layouts still fail before mutation. Thread offload_roots through the MoE dispatch call sites so parent-prefixed Accelerate hooks resolve, and drop the preflight rejection of 3D meta parameters (validate_offloaded_parameters already resolves them). Tests: fused 3D, bias, granular and selective-inversion projections on offloaded backing stores, bounded norm restoration, parent-prefixed hook resolution, quantized fail-closed, commit-failure rollback, and a full offline pipeline run on a disk-offloaded tiny Mixtral fixture that verifies the saved checkpoint carries the updated experts with no meta tensors. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>