# NVIDIA Jetson support plan Issue: https://github.com/elder-plinius/OBLITERATUS/issues/31 Internal testing assessment: [.aiwg/testing/jetson-support-assessment.md](../.aiwg/testing/jetson-support-assessment.md) ## Decision OBLITERATUS should treat Jetson AGX support as a distinct platform target, not as a generic Linux ARM build. Generic `linux/aarch64` packaging can prove that the Python package resolves and imports on ARM, but it does not prove CUDA on Jetson because Jetson devices depend on JetPack/L4T-matched CUDA, cuDNN, and PyTorch builds. The first supported target should be Jetson AGX Orin 64 GB on JetPack 6.x, run through either NVIDIA's Jetson PyTorch instructions or a JetPack-compatible container. Xavier-era JetPack 5 can be documented as best-effort until a runner exists. ## Evidence NVIDIA documents Jetson PyTorch as JetPack-specific pip wheels with GPU acceleration and cuDNN support, installed on top of the matching JetPack version: - https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html - https://catalog.ngc.nvidia.com/orgs/nvidia/containers/l4t-pytorch The NVIDIA NGC `l4t-pytorch` container is explicitly a Jetson/JetPack image and requires matching the container tag to the installed JetPack/L4T release. The `jetson-containers` project provides a practical composition path for PyTorch and Transformers on Jetson, including compatible autotag selection: - https://github.com/dusty-nv/jetson-containers PyTorch upstream support is moving: PyTorch 2.11 release notes say PyPI wheels now include CUDA 13.0 for Linux aarch64, while CUDA 12.6 and 12.8 wheels remain available from `download.pytorch.org`. That improves generic ARM CUDA, but it does not remove the need to match Jetson's installed JetPack/CUDA stack: - https://github.com/pytorch/pytorch/releases bitsandbytes currently publishes Linux aarch64 CUDA builds and lists targets for CUDA 11.8 through 13.x. AGX Orin's Ampere GPU should satisfy the compute capability requirement for 4-bit and 8-bit quantization, but the runtime still has to be validated on the Jetson image actually used by contributors: - https://huggingface.co/docs/bitsandbytes/en/installation ## Current repository gaps - `pyproject.toml` maps Linux and Windows Torch resolution to the CPU-only PyTorch index. That is correct for default CI, but it means the locked default environment will not discover Jetson CUDA. - `uv.lock` includes a Linux aarch64 bitsandbytes wheel, but Torch is locked as `2.13.0+cpu` on Linux. A Jetson runtime must intentionally override Torch after syncing the default lock, similar to the existing CUDA conditional lane. - `.github/workflows/conditional-tests.yml` has one CUDA lane labeled `self-hosted, linux, x64, cuda` and installs an official `cu130` Torch build. It cannot run on Jetson because the runner labels and Torch source are x64 CUDA-specific. - `docs/conditional-testing.md` documents generic CUDA and bitsandbytes probes, not Jetson labels, JetPack requirements, or container startup. - `Dockerfile` is based on `python:3.11-slim`, installs `requirements.txt`, and is documented as local-only. It is not a Jetson image and should not be used for a support claim. - `obliteratus/device.py` can already detect CUDA through PyTorch, so no special Jetson detection is needed for the first milestone. The blocker is installing a CUDA-enabled Jetson PyTorch runtime and proving the existing device and bitsandbytes contracts there. - `tests/conditional/test_cuda_runtime.py` is a good starting probe, but it does not record Jetson-specific platform evidence such as JetPack/L4T release, CUDA version, Python version, architecture, Torch build, and bitsandbytes binary availability. ## Implementation options ### Option A: Native Jetson runner, minimal repo changes Attach a self-hosted runner on the Jetson with labels such as `self-hosted`, `linux`, `ARM64`, `jetson`, and `cuda`. Add a new conditional gate `jetson-cuda-runtime` that: - syncs OBLITERATUS without replacing the system/vendor Jetson Torch build; - verifies `platform.machine()` is `aarch64`; - records `/etc/nv_tegra_release`, `nvcc --version`, `torch.__version__`, `torch.version.cuda`, `torch.cuda.get_device_name(0)`, and `bitsandbytes` import/runtime status; - runs the existing CUDA and bitsandbytes probes. Tradeoff: lowest abstraction and fastest to validate real hardware, but it requires maintaining the Jetson's host Python/runtime state carefully. ### Option B: Jetson container lane Base a Jetson-specific image on an NVIDIA L4T PyTorch image or build it with `jetson-containers` from `pytorch` plus `transformers`. Install OBLITERATUS into that image with dependency constraints that do not replace the container's Torch. Run the same `jetson-cuda-runtime` probe inside the container. Tradeoff: best reproducibility and closest to the reporter's concern about patched/static libraries, but the image has to track JetPack/L4T tags and may need per-JetPack dependency pins. ### Option C: Generic Linux ARM build plus separate Jetson UAT Add a hosted or self-hosted Linux ARM packaging job that installs OBLITERATUS with CPU Torch on aarch64, then keep Jetson CUDA as manual UAT evidence until a stable runner exists. Tradeoff: useful packaging signal for ARM contributors, but it must not be presented as Jetson GPU support because it does not exercise CUDA, cuDNN, device placement, quantization kernels, or JetPack compatibility. ## Recommended path Use Option B for the support claim and Option C as a cheap early-warning signal. The container path matches Jetson's dependency model, avoids polluting the default lock with Jetson-only Torch URLs, and gives contributors a repeatable recipe. A native runner can still be used to execute the container and collect evidence. Treat generic ARM64 as Tier 0/Tier 1 preflight only; require real-device Jetson smoke and release validation before publishing a support claim. ## Acceptance criteria - `docs/conditional-testing.md` documents Jetson as a separate conditional gate with supported JetPack versions, runner labels, and local/container commands. - `ci/conditional-test-policy.json` includes a `jetson-cuda-runtime` gate with a tracking issue and evidence freshness rule. - `.github/workflows/conditional-tests.yml` has a Jetson-selected job that runs only on a labeled Jetson runner or through a Jetson-compatible container. - The gate records JetPack/L4T, architecture, Torch CUDA version, CUDA device name, memory, and bitsandbytes status in JSON evidence. - The gate runs without skips: `python scripts/run_conditional_gate.py jetson-cuda-runtime`. - README support language says Jetson AGX Orin is supported only for the specific JetPack/runtime combinations that have fresh green conditional evidence. ## User guidance until support lands Do not install from the default OBLITERATUS lock and expect Jetson CUDA to work. Start from a JetPack-compatible PyTorch runtime, then install OBLITERATUS without replacing Torch. If using a container, prefer an L4T PyTorch image or a `jetson-containers` build that matches the device's JetPack/L4T release.