mirror of
https://github.com/elder-plinius/OBLITERATUS.git
synced 2026-08-29 22:20:36 +02:00
docs: specify Jetson support path
This commit is contained in:
Joseph Magly
@@ -84,7 +84,7 @@ Jetson CUDA support is tracked separately from this generic x64 CUDA lane. A
|
||||
generic Linux ARM build can prove package portability, but it does not prove
|
||||
Jetson GPU support because Jetson depends on a JetPack/L4T-matched CUDA, cuDNN,
|
||||
and PyTorch runtime. The support plan, recommended container path, and acceptance
|
||||
criteria are documented in [NVIDIA Jetson support plan](jetson-support.md).
|
||||
criteria are documented in [NVIDIA Jetson support plan](platforms/jetson.md).
|
||||
|
||||
## Apple MPS and MLX
|
||||
|
||||
|
||||
@@ -0,0 +1,136 @@
|
||||
# NVIDIA Jetson support plan
|
||||
|
||||
Issue: https://github.com/elder-plinius/OBLITERATUS/issues/31
|
||||
|
||||
## Status
|
||||
|
||||
Native Jetson AGX support is not claimed yet. OBLITERATUS should treat Jetson
|
||||
as a dedicated conditional runtime lane, not as part of the default pull-request
|
||||
gate and not as a generic Linux ARM build.
|
||||
|
||||
The current OBLITERATUS CUDA path delegates discovery to PyTorch through
|
||||
`torch.cuda.is_available()`. If a Jetson AGX host reports no CUDA inside
|
||||
OBLITERATUS, the first thing to verify is the JetPack/L4T/PyTorch/container
|
||||
stack, because NVIDIA publishes Jetson-specific PyTorch builds intended for
|
||||
specified JetPack versions.
|
||||
|
||||
## Decision
|
||||
|
||||
Support Jetson through a JetPack-pinned runtime contract:
|
||||
|
||||
- Keep ordinary PR CI CPU-only, offline, and architecture-neutral.
|
||||
- Add a Jetson conditional gate once a Jetson runner is available.
|
||||
- Prefer an NVIDIA-supported Jetson PyTorch container or NVIDIA Jetson PyTorch
|
||||
wheel for the exact JetPack release under test.
|
||||
- Do not use the existing x64 CUDA gate as Jetson evidence.
|
||||
- Do not treat a generic `linux/arm64` build as evidence that CUDA works on
|
||||
Jetson.
|
||||
|
||||
## Why generic ARM is insufficient
|
||||
|
||||
Jetson support couples at least five moving pieces:
|
||||
|
||||
- Jetson hardware family and compute capability.
|
||||
- JetPack version.
|
||||
- Jetson Linux/L4T version and Ubuntu base image.
|
||||
- CUDA, cuDNN, TensorRT, and related NVIDIA libraries.
|
||||
- PyTorch build or container version.
|
||||
|
||||
NVIDIA's Jetson PyTorch documentation says the PyTorch packages are installed
|
||||
on top of a specified JetPack version, and the compatibility table maps PyTorch
|
||||
versions to NVIDIA framework containers/wheels and JetPack versions. A generic
|
||||
ARM build can prove that Python code imports on `aarch64`; it cannot prove that
|
||||
CUDA, cuDNN, TensorRT, or PyTorch CUDA dispatch works on Jetson.
|
||||
|
||||
## Initial support matrix
|
||||
|
||||
Start with the hardware reported in issue #31: Jetson AGX devices with 64 GB
|
||||
unified memory.
|
||||
|
||||
Recommended first support tier:
|
||||
|
||||
| Tier | Hardware | JetPack | OS / CUDA baseline | Evidence requirement |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Target | Jetson AGX Orin 64 GB | 6.2 | Jetson Linux 36.4.3 / CUDA 12.6 | Native Jetson runner or NVIDIA Jetson container on Jetson hardware |
|
||||
| Evaluate | Jetson AGX Thor | 7.x | Jetson Linux 38/39 / Ubuntu 24.04 / CUDA 13.x family | Separate runner and issue before claiming support |
|
||||
| Legacy | Jetson AGX Xavier | 5.1.x | Jetson Linux 35.x / Ubuntu 20.04 / CUDA 11.x family | Defer unless a maintainer/user provides hardware and demand |
|
||||
|
||||
Do not collapse these tiers into one "ARM64" support claim.
|
||||
|
||||
## Installation shape
|
||||
|
||||
The generic local Dockerfile uses `python:3.11-slim` and is not the Jetson
|
||||
runtime image. A Jetson runtime should use one of these approaches:
|
||||
|
||||
1. Start from an NVIDIA Jetson-compatible PyTorch framework container for the
|
||||
selected JetPack version, then install OBLITERATUS without replacing the
|
||||
container's validated PyTorch stack.
|
||||
2. On a flashed Jetson host, install the NVIDIA Jetson PyTorch wheel matching
|
||||
the installed JetPack release, then install OBLITERATUS in a virtual
|
||||
environment without allowing dependency resolution to replace `torch`.
|
||||
|
||||
The lock policy should make the Jetson torch source explicit. The existing
|
||||
Linux PR lock intentionally uses CPU-only PyTorch. A Jetson install path needs
|
||||
an override or separate constraints file that preserves NVIDIA's Jetson PyTorch
|
||||
runtime.
|
||||
|
||||
## Conditional gate
|
||||
|
||||
Add a new gate instead of modifying the x64 CUDA gate:
|
||||
|
||||
- Gate id: `jetson-runtime`
|
||||
- Runner labels: `self-hosted`, `linux`, `ARM64`, `jetson`
|
||||
- Optional labels by tier: `orin`, `jetpack-6` or `thor`, `jetpack-7`
|
||||
- Trigger: manual dispatch and release/scheduled validation only
|
||||
- Evidence retention: same 30-day conditional-evidence policy as other hardware
|
||||
gates
|
||||
|
||||
The gate should verify:
|
||||
|
||||
- `platform.machine()` is `aarch64` or equivalent ARM64.
|
||||
- `torch.cuda.is_available()` is true.
|
||||
- `torch.version.cuda` is not `None`.
|
||||
- `torch.cuda.get_device_name(0)` identifies the Jetson GPU class.
|
||||
- OBLITERATUS resolves `device=auto` to `cuda`.
|
||||
- A small CUDA tensor operation completes with finite output.
|
||||
- The existing offloaded-surgery CUDA probe passes.
|
||||
- `bitsandbytes` NF4/4-bit quantization is either proven on that exact Jetson
|
||||
stack or documented as unsupported for the tier.
|
||||
- A tiny Hugging Face model run passes only when the model-download gate is
|
||||
explicitly selected and the runner has the required account/cache policy.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
Jetson support can be claimed for a tier only after all of the following are
|
||||
true:
|
||||
|
||||
- The supported Jetson module and JetPack version are named in this document.
|
||||
- The install instructions pin the JetPack-compatible PyTorch container or
|
||||
wheel source.
|
||||
- The Jetson conditional gate produces non-skipped green evidence on the exact
|
||||
commit being claimed.
|
||||
- The release notes distinguish generic ARM importability from Jetson CUDA
|
||||
support.
|
||||
- The docs state memory expectations for 64 GB unified memory and recommend
|
||||
small models for validation before large ablation runs.
|
||||
|
||||
## Operational notes
|
||||
|
||||
Jetson's unified memory is shared by the OS, CUDA, model weights, activations,
|
||||
and file cache. Treat "64 GB" as a capacity class, not guaranteed usable model
|
||||
memory. Use small models for smoke tests, then move larger GPU validation to
|
||||
dedicated CUDA hosts such as Titan when those resources are available.
|
||||
|
||||
Logging into a Hugging Face account is expected only for gated/private models,
|
||||
license-gated models, or rate-limit avoidance. It should not be required for the
|
||||
offline CPU PR gate or for the Jetson CUDA hardware probe. Any model-download
|
||||
validation must remain an explicit conditional gate.
|
||||
|
||||
## Sources
|
||||
|
||||
- `REF-JETSON-PYTORCH-INSTALL`: NVIDIA, Installing PyTorch for Jetson Platform.
|
||||
- `REF-JETSON-PYTORCH-RELEASES`: NVIDIA, PyTorch for Jetson Platform release notes.
|
||||
- `REF-JETPACK-62`: NVIDIA, JetPack 6.2 release notes.
|
||||
- `REF-JETPACK-7-DOWNLOADS`: NVIDIA, JetPack SDK downloads and notes.
|
||||
- `REF-UV-PYTORCH`: Astral, Using uv with PyTorch.
|
||||
- `REF-BITSANDBYTES-INSTALL`: Hugging Face, bitsandbytes installation guide.
|
||||
Reference in New Issue
Block a user