OBLITERATUS/docs/jetson-support.md

# NVIDIA Jetson support plan

Issue: https://github.com/elder-plinius/OBLITERATUS/issues/31

Internal testing assessment:
[.aiwg/testing/jetson-support-assessment.md](../.aiwg/testing/jetson-support-assessment.md)

## Decision

OBLITERATUS should treat Jetson AGX support as a distinct platform target, not
as a generic Linux ARM build. Generic `linux/aarch64` packaging can prove that
the Python package resolves and imports on ARM, but it does not prove CUDA on
Jetson because Jetson devices depend on JetPack/L4T-matched CUDA, cuDNN, and
PyTorch builds.

The first supported target should be Jetson AGX Orin 64 GB on JetPack 6.x, run
through either NVIDIA's Jetson PyTorch instructions or a JetPack-compatible
container. Xavier-era JetPack 5 can be documented as best-effort until a runner
exists.

## Evidence

NVIDIA documents Jetson PyTorch as JetPack-specific pip wheels with GPU
acceleration and cuDNN support, installed on top of the matching JetPack
version:

- https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html
- https://catalog.ngc.nvidia.com/orgs/nvidia/containers/l4t-pytorch

The NVIDIA NGC `l4t-pytorch` container is explicitly a Jetson/JetPack image and
requires matching the container tag to the installed JetPack/L4T release. The
`jetson-containers` project provides a practical composition path for PyTorch
and Transformers on Jetson, including compatible autotag selection:

- https://github.com/dusty-nv/jetson-containers

PyTorch upstream support is moving: PyTorch 2.11 release notes say PyPI wheels
now include CUDA 13.0 for Linux aarch64, while CUDA 12.6 and 12.8 wheels remain
available from `download.pytorch.org`. That improves generic ARM CUDA, but it
does not remove the need to match Jetson's installed JetPack/CUDA stack:

- https://github.com/pytorch/pytorch/releases

bitsandbytes currently publishes Linux aarch64 CUDA builds and lists targets
for CUDA 11.8 through 13.x. AGX Orin's Ampere GPU should satisfy the compute
capability requirement for 4-bit and 8-bit quantization, but the runtime still
has to be validated on the Jetson image actually used by contributors:

- https://huggingface.co/docs/bitsandbytes/en/installation

## Current repository gaps

- `pyproject.toml` maps Linux and Windows Torch resolution to the CPU-only
  PyTorch index. That is correct for default CI, but it means the locked default
  environment will not discover Jetson CUDA.
- `uv.lock` includes a Linux aarch64 bitsandbytes wheel, but Torch is locked as
  `2.13.0+cpu` on Linux. A Jetson runtime must intentionally override Torch
  after syncing the default lock, similar to the existing CUDA conditional lane.
- `.github/workflows/conditional-tests.yml` has one CUDA lane labeled
  `self-hosted, linux, x64, cuda` and installs an official `cu130` Torch build.
  It cannot run on Jetson because the runner labels and Torch source are x64
  CUDA-specific.
- `docs/conditional-testing.md` documents generic CUDA and bitsandbytes probes,
  not Jetson labels, JetPack requirements, or container startup.
- `Dockerfile` is based on `python:3.11-slim`, installs `requirements.txt`, and
  is documented as local-only. It is not a Jetson image and should not be used
  for a support claim.
- `obliteratus/device.py` can already detect CUDA through PyTorch, so no special
  Jetson detection is needed for the first milestone. The blocker is installing
  a CUDA-enabled Jetson PyTorch runtime and proving the existing device and
  bitsandbytes contracts there.
- `tests/conditional/test_cuda_runtime.py` is a good starting probe, but it does
  not record Jetson-specific platform evidence such as JetPack/L4T release,
  CUDA version, Python version, architecture, Torch build, and bitsandbytes
  binary availability.

## Implementation options

### Option A: Native Jetson runner, minimal repo changes

Attach a self-hosted runner on the Jetson with labels such as
`self-hosted`, `linux`, `ARM64`, `jetson`, and `cuda`. Add a new conditional
gate `jetson-cuda-runtime` that:

- syncs OBLITERATUS without replacing the system/vendor Jetson Torch build;
- verifies `platform.machine()` is `aarch64`;
- records `/etc/nv_tegra_release`, `nvcc --version`, `torch.__version__`,
  `torch.version.cuda`, `torch.cuda.get_device_name(0)`, and
  `bitsandbytes` import/runtime status;
- runs the existing CUDA and bitsandbytes probes.

Tradeoff: lowest abstraction and fastest to validate real hardware, but it
requires maintaining the Jetson's host Python/runtime state carefully.

### Option B: Jetson container lane

Base a Jetson-specific image on an NVIDIA L4T PyTorch image or build it with
`jetson-containers` from `pytorch` plus `transformers`. Install OBLITERATUS into
that image with dependency constraints that do not replace the container's
Torch. Run the same `jetson-cuda-runtime` probe inside the container.

Tradeoff: best reproducibility and closest to the reporter's concern about
patched/static libraries, but the image has to track JetPack/L4T tags and may
need per-JetPack dependency pins.

### Option C: Generic Linux ARM build plus separate Jetson UAT

Add a hosted or self-hosted Linux ARM packaging job that installs OBLITERATUS
with CPU Torch on aarch64, then keep Jetson CUDA as manual UAT evidence until a
stable runner exists.

Tradeoff: useful packaging signal for ARM contributors, but it must not be
presented as Jetson GPU support because it does not exercise CUDA, cuDNN,
device placement, quantization kernels, or JetPack compatibility.

## Recommended path

Use Option B for the support claim and Option C as a cheap early-warning signal.
The container path matches Jetson's dependency model, avoids polluting the
default lock with Jetson-only Torch URLs, and gives contributors a repeatable
recipe. A native runner can still be used to execute the container and collect
evidence. Treat generic ARM64 as Tier 0/Tier 1 preflight only; require
real-device Jetson smoke and release validation before publishing a support
claim.

## Acceptance criteria

- `docs/conditional-testing.md` documents Jetson as a separate conditional gate
  with supported JetPack versions, runner labels, and local/container commands.
- `ci/conditional-test-policy.json` includes a `jetson-cuda-runtime` gate with a
  tracking issue and evidence freshness rule.
- `.github/workflows/conditional-tests.yml` has a Jetson-selected job that runs
  only on a labeled Jetson runner or through a Jetson-compatible container.
- The gate records JetPack/L4T, architecture, Torch CUDA version, CUDA device
  name, memory, and bitsandbytes status in JSON evidence.
- The gate runs without skips:
  `python scripts/run_conditional_gate.py jetson-cuda-runtime`.
- README support language says Jetson AGX Orin is supported only for the
  specific JetPack/runtime combinations that have fresh green conditional
  evidence.

## User guidance until support lands

Do not install from the default OBLITERATUS lock and expect Jetson CUDA to work.
Start from a JetPack-compatible PyTorch runtime, then install OBLITERATUS
without replacing Torch. If using a container, prefer an L4T PyTorch image or a
`jetson-containers` build that matches the device's JetPack/L4T release.