mirror of
https://github.com/elder-plinius/OBLITERATUS.git
synced 2026-08-17 16:37:30 +02:00
fix: harden quantized checkpoint integration
This commit is contained in:
+23
-12
@@ -1821,10 +1821,13 @@ protocol; anything here that surprised us is marked **(finding)**.
|
||||
detection must accept both `_scale`/`_scale_2` suffixes and the dotted
|
||||
`.weight_scale` family **(finding: first state-dict test run failed on
|
||||
exactly this)**.
|
||||
4. **compressed-tensors stores reciprocal scales.** Dequant divides rather
|
||||
than multiplies. Round-tripping reciprocals through FP8-E4M3 storage is
|
||||
measurably coarser (cosine ~0.986 vs ~0.999 for direct storage on random
|
||||
weights) — expected, not a bug.
|
||||
4. **compressed-tensors inverts only the global NVFP4 scale.** Its documented
|
||||
converter maps ModelOpt `weight` to `weight_packed`, leaves `weight_scale`
|
||||
unchanged, and maps `weight_scale_2` to the reciprocal
|
||||
`weight_global_scale`. Dequantization therefore multiplies by the FP8 block
|
||||
scale and divides by the global scale. Treating both scales as reciprocal
|
||||
produced a self-consistent synthetic test but did not match the published
|
||||
checkpoint contract **(maintainer audit finding)**.
|
||||
5. **torch 2.13.0+cpu**: `torch.float4_e2m1fn_x2` exists but the runtime
|
||||
upcast probe fails on this CPU build, so the manual nibble-LUT path is
|
||||
what actually runs there; the probe design (rather than a version check)
|
||||
@@ -1833,9 +1836,11 @@ protocol; anything here that surprised us is marked **(finding)**.
|
||||
so the FP8/NVFP4 loud-failure guard in `_dequantize_weight` must run
|
||||
*before* the bitsandbytes branch or the error message is misleading
|
||||
**(finding: unit test caught the wrong branch firing)**.
|
||||
7. NVFP4 round-trip error on random Gaussian weights: relative Frobenius
|
||||
error ~0.10 direct / ~0.18 reciprocal, cosine >0.98 — this is the
|
||||
intrinsic precision of the format, not implementation error.
|
||||
7. NVFP4 round-trip error on the deterministic random-Gaussian fixtures is
|
||||
below 0.15 relative Frobenius error with cosine above 0.995 for both
|
||||
ModelOpt and compressed-tensors global-scale storage. Seeds, construction,
|
||||
and thresholds are recorded in
|
||||
`tests/fixtures/quant_dequant_provenance.json`.
|
||||
8. **Real checkpoints are mixed-precision.** `nvidia/Nemotron-3-Nano-Omni-
|
||||
30B-A3B-Reasoning-NVFP4` ships `quant_method: modelopt` with
|
||||
`quant_algo: MIXED_PRECISION`: FP8 (8-bit float, per-tensor
|
||||
@@ -1857,7 +1862,12 @@ Smoke-test results on real checkpoints: see Appendix E continuation below
|
||||
|
||||
---
|
||||
|
||||
## Appendix E2: Nemotron-3-Nano-Omni smoke runs (2026-07-24, 1x A100-80GB)
|
||||
## Appendix E2: Contributor-reported Nemotron-3-Nano-Omni smoke runs (2026-07-24)
|
||||
|
||||
The following A100 results were supplied with the contribution. They do not
|
||||
include immutable checkpoint revisions or retained machine-verifiable logs and
|
||||
were not independently reproduced during maintainer integration. They are
|
||||
historical engineering notes, not release evidence or a support claim.
|
||||
|
||||
Real-checkpoint validation of the FP8/NVFP4 dequantization path. Both
|
||||
targets loaded with **zero quantized tensors left, zero NaNs, and
|
||||
@@ -1944,10 +1954,11 @@ Notes:
|
||||
- The compressed-tensors per-channel FP8 dense alternate
|
||||
(`RedHatAI/Llama-3.1-8B-Instruct-FP8-dynamic`) was not run: the exact
|
||||
code path (`dequantize_fp8_per_channel`, direct scales) was already
|
||||
exercised by the Nemotron FP8 checkpoint, and Llama-3.1 is a gated
|
||||
repo. CT-NVFP4 (reciprocal scales) is covered by synthetic unit tests
|
||||
only — no public checkpoint was needed after the Nemotron layout
|
||||
parsed cleanly.
|
||||
exercised by the contributor's Nemotron FP8 run, and Llama-3.1 is a gated
|
||||
repo. CT-NVFP4 is covered by deterministic synthetic contract tests only;
|
||||
those tests follow the published `weight_packed`/`weight_scale`/
|
||||
`weight_global_scale` mapping. A pinned public-checkpoint run remains
|
||||
necessary before making a runtime support claim.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user