docs: harden Jetson runtime decision

This commit is contained in:
Joseph Magly
2026-08-21 19:40:38 -04:00
parent 8814dbe359
commit 1e870ccee0
9 changed files with 168 additions and 246 deletions
+29 -17
View File
@@ -42,17 +42,19 @@ versions to NVIDIA framework containers/wheels and JetPack versions. A generic
ARM build can prove that Python code imports on `aarch64`; it cannot prove that
CUDA, cuDNN, TensorRT, or PyTorch CUDA dispatch works on Jetson.
## Initial support matrix
## Proposed initial support matrix
Start with the hardware reported in issue #31: Jetson AGX devices with 64 GB
unified memory.
Recommended first support tier:
This is a planning target, not a current support claim. The implementation must
replace the JetPack family with the exact patch installed on the available
runner before publishing compatibility:
| Tier | Hardware | JetPack | OS / CUDA baseline | Evidence requirement |
| --- | --- | --- | --- | --- |
| Target | Jetson AGX Orin 64 GB | 6.2 | Jetson Linux 36.4.3 / CUDA 12.6 | Native Jetson runner or NVIDIA Jetson container on Jetson hardware |
| Evaluate | Jetson AGX Thor | 7.x | Jetson Linux 38/39 / Ubuntu 24.04 / CUDA 13.x family | Separate runner and issue before claiming support |
| Initial candidate | Jetson AGX Orin 64 GB | 6.2.x, exact patch TBD | L4T/CUDA values from the selected patch | Native Jetson runner or pinned compatible container on Jetson hardware |
| Evaluate later | Jetson AGX Thor | 7.x, exact release TBD | Select only after NVIDIA's PyTorch compatibility table covers the release | Separate runner and evidence before claiming support |
| Legacy | Jetson AGX Xavier | 5.1.x | Jetson Linux 35.x / Ubuntu 20.04 / CUDA 11.x family | Defer unless a maintainer/user provides hardware and demand |
Do not collapse these tiers into one "ARM64" support claim.
@@ -72,7 +74,10 @@ runtime image. A Jetson runtime should use one of these approaches:
The lock policy should make the Jetson torch source explicit. The existing
Linux PR lock intentionally uses CPU-only PyTorch. A Jetson install path needs
an override or separate constraints file that preserves NVIDIA's Jetson PyTorch
runtime.
runtime. It must also prevent the generic PyPI Linux-aarch64 bitsandbytes wheel
from being selected: upstream documents that wheel as SBSA/server ARM and says
Jetson L4T/JetPack requires a source build. Until a pinned source build passes
on the selected device, bitsandbytes is unsupported for that tier.
## Conditional gate
@@ -85,6 +90,9 @@ Add a new gate instead of modifying the x64 CUDA gate:
- Evidence retention: same 30-day conditional-evidence policy as other hardware
gates
The job must run only from a trusted ref or reviewed maintainer dispatch. A
persistent self-hosted Jetson must never execute untrusted pull-request code.
The gate should verify:
- `platform.machine()` is `aarch64` or equivalent ARM64.
@@ -94,8 +102,8 @@ The gate should verify:
- OBLITERATUS resolves `device=auto` to `cuda`.
- A small CUDA tensor operation completes with finite output.
- The existing offloaded-surgery CUDA probe passes.
- `bitsandbytes` NF4/4-bit quantization is either proven on that exact Jetson
stack or documented as unsupported for the tier.
- A pinned, source-built `bitsandbytes` NF4/4-bit path is proven on that exact
Jetson stack, or bitsandbytes is documented as unsupported for the tier.
- A tiny Hugging Face model run passes only when the model-download gate is
explicitly selected and the runner has the required account/cache policy.
@@ -109,6 +117,9 @@ true:
wheel source.
- The Jetson conditional gate produces non-skipped green evidence on the exact
commit being claimed.
- The named `jetson-runtime` workflow job uses the documented runner labels and
retains `conditional-jetson-<run-attempt>` logs and environment metadata for
30 days.
- The release notes distinguish generic ARM importability from Jetson CUDA
support.
- The docs state memory expectations for 64 GB unified memory and recommend
@@ -121,16 +132,17 @@ and file cache. Treat "64 GB" as a capacity class, not guaranteed usable model
memory. Use small models for smoke tests, then move larger GPU validation to
dedicated CUDA hosts such as Titan when those resources are available.
Logging into a Hugging Face account is expected only for gated/private models,
license-gated models, or rate-limit avoidance. It should not be required for the
offline CPU PR gate or for the Jetson CUDA hardware probe. Any model-download
validation must remain an explicit conditional gate.
The Jetson hardware probe does not require a Hugging Face login. Model download
testing remains the separate, explicitly selected `model-download-runtime`
gate; credentials are relevant only when that selected model itself requires
them.
## Sources
- `REF-JETSON-PYTORCH-INSTALL`: NVIDIA, Installing PyTorch for Jetson Platform.
- `REF-JETSON-PYTORCH-RELEASES`: NVIDIA, PyTorch for Jetson Platform release notes.
- `REF-JETPACK-62`: NVIDIA, JetPack 6.2 release notes.
- `REF-JETPACK-7-DOWNLOADS`: NVIDIA, JetPack SDK downloads and notes.
- `REF-UV-PYTORCH`: Astral, Using uv with PyTorch.
- `REF-BITSANDBYTES-INSTALL`: Hugging Face, bitsandbytes installation guide.
- [NVIDIA: Installing PyTorch for Jetson Platform](https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html)
- [NVIDIA: PyTorch for Jetson compatibility table](https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform-release-notes/pytorch-jetson-rel.html)
- [NVIDIA: JetPack 6.2 release notes](https://docs.nvidia.com/jetson/archives/jetpack-archived/jetpack-62/release-notes/index.html)
- [NVIDIA: current JetPack downloads and notes](https://developer.nvidia.com/embedded/jetpack/downloads)
- [Astral: Using uv with PyTorch](https://docs.astral.sh/uv/guides/integration/pytorch/)
- [Hugging Face: bitsandbytes installation guide](https://huggingface.co/docs/bitsandbytes/installation)
- [GitHub: secure use of self-hosted runners](https://docs.github.com/en/actions/reference/security/secure-use#hardening-for-self-hosted-runners)