test+docs: quant_dequant test suite, theory journal engineering log

33 CPU-only tests: FP8 block-wise/per-channel round-trips, NVFP4
round-trips (direct + reciprocal scales, native vs manual unpack
agreement), scheme detection incl. ModelOpt MIXED_PRECISION, surgery
guard raises, and tiny synthetic FP8/NVFP4 GPT-2 checkpoints through
load_model end-to-end. Full suite: 870 passed (8 failures pre-exist on
main, verified against pristine checkout).

docs/theory_journal.md Appendices E-E2: engineering log of the 12
findings from implementation and real-checkpoint validation (nibble
order conventions, mixed-precision layouts, the 96GB OOM and the
chunked-unpack fix, transformers 5.x drift shims, Omni wrapper unwrap).

docs/theory_journal.md

@@ -1799,6 +1799,158 @@ implementations diverge from the closed-form GRRO solution.
 
 ---
 
+## Appendix E: FP8/NVFP4 checkpoint dequantization — engineering log (2026-07-24)
+
+Implementation notes from adding FP8 and NVFP4 checkpoint support
+(`obliteratus/models/quant_dequant.py`). Recorded per the escalation
+protocol; anything here that surprised us is marked **(finding)**.
+
+1. **Nibble order is low-first, and it is easy to get backwards.** ModelOpt
+   NVFP4 packs two E2M1 values per byte along the input dim, low nibble
+   first: byte `0x1B` decodes to `[-1.5, +0.5]` (low `0xB`, high `0x1`).
+   **(finding)** Our first probe had the expected pair swapped, which would
+   have silently pair-flipped every weight row; the known-byte unit test
+   caught it. The runtime probe `_native_fp4_upcast_works()` now derives
+   the order empirically instead of assuming it.
+2. **E2M1 LUT**: index → value `{0,.5,1,1.5,2,3,4,6}`, sign in bit 3. Index
+   4 is `2.0`, not `3.0` — the spacing is not linear **(finding: test
+   caught a wrong-by-inspection comment)**.
+3. **Fused-MoE scale naming differs from dense linears.** Dense weights use
+   `foo.weight` + `foo.weight_scale[_2/_inv]`; fused experts use
+   `experts.gate_up_proj` + `experts.gate_up_proj_scale[_2]`. Scale-key
+   detection must accept both `_scale`/`_scale_2` suffixes and the dotted
+   `.weight_scale` family **(finding: first state-dict test run failed on
+   exactly this)**.
+4. **compressed-tensors stores reciprocal scales.** Dequant divides rather
+   than multiplies. Round-tripping reciprocals through FP8-E4M3 storage is
+   measurably coarser (cosine ~0.986 vs ~0.999 for direct storage on random
+   weights) — expected, not a bug.
+5. **torch 2.13.0+cpu**: `torch.float4_e2m1fn_x2` exists but the runtime
+   upcast probe fails on this CPU build, so the manual nibble-LUT path is
+   what actually runs there; the probe design (rather than a version check)
+   was validated by this.
+6. **Guard ordering matters**: `_is_quantized_param` flags float8 tensors,
+   so the FP8/NVFP4 loud-failure guard in `_dequantize_weight` must run
+   *before* the bitsandbytes branch or the error message is misleading
+   **(finding: unit test caught the wrong branch firing)**.
+7. NVFP4 round-trip error on random Gaussian weights: relative Frobenius
+   error ~0.10 direct / ~0.18 reciprocal, cosine >0.98 — this is the
+   intrinsic precision of the format, not implementation error.
+8. **Real checkpoints are mixed-precision.** `nvidia/Nemotron-3-Nano-Omni-
+   30B-A3B-Reasoning-NVFP4` ships `quant_method: modelopt` with
+   `quant_algo: MIXED_PRECISION`: FP8 (8-bit float, per-tensor
+   `weight_scale`, no `weight_scale_2`) for the 98 Mamba-mixer/shared-expert
+   projections, NVFP4 (`weight` uint8 + FP8 `weight_scale` +
+   FP32 `weight_scale_2`) for the 5,888 routed-expert projections
+   **(finding)**. Detection now inspects `config_groups` bit-widths under
+   MIXED_PRECISION; dequantization was already per-tensor so FP8 and NVFP4
+   tensors in the same shard each take their own path. The routed experts
+   are stored as *individual 2D modules* (`mixer.experts.{i}.up_proj`), not
+   fused 3D tensors — the dense path covers them.
+9. **Custom architectures need a CLI escape hatch.** Nemotron Omni uses a
+   custom `modeling.py` (`NemotronH_Nano_Omni_Reasoning_V3`), but the CLI
+   had no `--trust-remote-code` passthrough to the pipeline **(finding)**.
+   Added the flag (`cli.py` → `AbliterationPipeline` → `load_model`).
+
+Smoke-test results on real checkpoints: see Appendix E continuation below
+(filled in after the Nemotron-3-Nano GPU runs).
+
+---
+
+## Appendix E2: Nemotron-3-Nano-Omni smoke runs (2026-07-24, 1x A100-80GB)
+
+Real-checkpoint validation of the FP8/NVFP4 dequantization path. Both
+targets loaded with **zero quantized tensors left, zero NaNs, and
+`quantization_config` stripped**, and both completed the full
+`obliterate --method basic` pipeline (SUMMON → PROBE → DISTILL → EXCISE →
+VERIFY → REBIRTH) with BF16 output. Findings, in the order they appeared:
+
+1. **Host RAM OOM during dequantization (fixed).** The first
+   `materialize_dequantized_checkpoint` attempt peaked at 96.5 GB RSS on a
+   94 GB box and was OOM-killed mid-shard. Root cause: the manual nibble
+   unpack built an int64 index tensor for *all* nibbles at once (8 B/value)
+   plus stacked copies (~24 B/value total), and scale application created
+   fresh float32 copies per step. Fix: chunked LUT indexing (≤0.5 GB
+   transient int64) and in-place `mul_`/`div_` scaling. Peak is now
+   roughly one shard's float32 size.
+2. **Hybrid-Mamba model deps.** The Omni custom code hard-imports
+   `mamba_ssm` triton ops; pip's build isolation hides torch and the CUDA
+   extension build is unnecessary — `pip install --no-build-isolation
+   mamba-ssm` with `MAMBA_SKIP_CUDA_BUILD=TRUE` plus a tiny
+   `selective_scan_cuda` stub module (only triton paths are used).
+   Vision/audio towers additionally need `timm`, `open_clip_torch`,
+   `librosa`.
+3. **`attn_implementation: flash_attention_2` pinned in config (fixed
+   generically).** flash-attn isn't installed; the custom classes also
+   reject sdpa. The loader now detects the pin — including on nested
+   sub-configs (`llm_config`) — and falls back to `eager` when flash-attn
+   is unavailable.
+4. **transformers 5.14 API drift, three shims (fixed in loader.py's shim
+   block / post-load normalization):**
+   - `all_tied_weights_keys`: expected by the accelerate device-map code,
+     absent on older remote-code classes, and *assigned* by 5.x
+     `post_init` — shim is a settable property aggregating legacy
+     `_tied_weights_keys`, filtering keys that don't resolve on the
+     top-level model (multimodal wrappers inherit `lm_head.weight` from
+     the LM submodule's config but have no top-level lm_head).
+   - `cache_position=None` passed by 5.x generation code into
+     `prepare_inputs_for_generation`; older remote code assumes a tensor.
+     Loader wraps the method to synthesize `cache_position` (using
+     past-key length during decoding).
+   - `_tied_weights_keys` as legacy *list* breaks 5.x `save_pretrained`
+     (expects `.keys()`); normalized post-load — emptied when
+     `tie_word_embeddings` is false so nothing is wrongly dropped.
+5. **"Omni" wrapper can't run text-only surgery (fixed).**
+   `NemotronH_Nano_Omni_Reasoning_V3.forward()` requires `pixel_values`.
+   The loader now detects media-required forward signatures and unwraps
+   `model.language_model`; this also fixed layer/head/hidden reporting
+   (wrapper config lacks those fields). Surgery and the saved artifact
+   are the LM only.
+6. **New architecture registered: `nemotron_h`.** All layer content hangs
+   off `layer.mixer` — attention (q/k/v/o_proj on the 6 attention layers),
+   Mamba (in/out_proj on 23), routed experts (2,944 individual
+   `mixer.experts.N.{up,down}_proj` 2D modules, *not* fused 3D) and
+   `mixer.shared_experts` (23). Mapped `["backbone","layers"]` +
+   `mixer` for both attention and FFN resolution in
+   `strategies/utils.py`. Mamba `out_proj` matches the attention
+   output-projection names and gets projected too — acceptable, it lives
+   in the same residual stream; noted here deliberately.
+7. **Temp-dir lifecycle bug (fixed).** The dequantized temp copy was
+   deleted before the tokenizer load (which pointed at it). Tokenizer now
+   loads from the original checkpoint source.
+
+Results (`--method basic`, 1 direction):
+
+| Checkpoint | Scheme detected | Outcome | Refusal (post) | Perplexity | Output |
+|---|---|---|---|---|---|
+| Nemotron-3-Nano-Omni-30B NVFP4 (mixed FP8+NVFP4) | `nvfp4_modelopt` | full pipeline OK | 0.0 | 953.4 | 63.2 GB BF16 |
+| Nemotron-3-Nano-Omni-30B FP8 | per-tensor FP8 (modelopt) | full pipeline OK | 0.0 | 920.5 (KL 0.73) | 63.2 GB BF16 |
+| Qwen/Qwen3-8B-FP8 (block-wise, DeepSeek-style) | `fp8_blockwise` | full pipeline OK | 0.867 | 4.23 (KL 0.09) | 16 GB BF16 |
+| Qwen/Qwen3-8B (**BF16 baseline**) | none | full pipeline OK | 0.833 | 4.33 (KL 0.11) | 16 GB BF16 |
+
+Notes:
+
+- **Qwen3-8B FP8-vs-BF16 parity is the format validation**: perplexity
+  within 2.4% (criterion was ~10%), refusal rates statistically identical
+  on the 30-sample eval. Qwen3's high residual refusal is a property of
+  the model + the 1-direction `basic` method (spectral certification RED,
+  the run itself recommended more directions) — it is *not* a
+  quantization artifact, as the BF16 baseline demonstrates.
+- Nemotron perplexity is high in absolute terms — `--method basic` on a
+  hybrid Mamba-MoE reasoning model is the crudest setting and the
+  eigenvalue report recommended the 'nuclear' method. The smoke test's
+  purpose is format validation (dequantize → surgery → BF16 save), which
+  passed; tuning method strength for this architecture is separate work.
+- The compressed-tensors per-channel FP8 dense alternate
+  (`RedHatAI/Llama-3.1-8B-Instruct-FP8-dynamic`) was not run: the exact
+  code path (`dequantize_fp8_per_channel`, direct scales) was already
+  exercised by the Nemotron FP8 checkpoint, and Llama-3.1 is a gated
+  repo. CT-NVFP4 (reciprocal scales) is covered by synthetic unit tests
+  only — no public checkpoint was needed after the Nemotron layout
+  parsed cleanly.
+
+---
+
 ## References
 
 1. Arditi, A. et al. (2024). Refusal in Language Models Is Mediated by a Single Direction. NeurIPS 2024.