CalvinBackup/OBLITERATUS
1
0
Fork 0
mirror of https://github.com/elder-plinius/OBLITERATUS.git synced 2026-08-30 06:30:37 +02:00
Code Issues Packages Projects Releases Wiki Activity

docs: harden Jetson runtime decision

Browse Source
This commit is contained in:
Joseph Magly
2026-08-21 19:40:38 -04:00
parent 8814dbe359
commit 1e870ccee0
9 changed files with 168 additions and 246 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/jetson-support.md
-147
View File
@@ -1,147 +0,0 @@
# NVIDIA Jetson support plan
Issue: https://github.com/elder-plinius/OBLITERATUS/issues/31
Internal testing assessment:
[.aiwg/testing/jetson-support-assessment.md](../.aiwg/testing/jetson-support-assessment.md)
## Decision
OBLITERATUS should treat Jetson AGX support as a distinct platform target, not
as a generic Linux ARM build. Generic `linux/aarch64` packaging can prove that
the Python package resolves and imports on ARM, but it does not prove CUDA on
Jetson because Jetson devices depend on JetPack/L4T-matched CUDA, cuDNN, and
PyTorch builds.
The first supported target should be Jetson AGX Orin 64 GB on JetPack 6.x, run
through either NVIDIA's Jetson PyTorch instructions or a JetPack-compatible
container. Xavier-era JetPack 5 can be documented as best-effort until a runner
exists.
## Evidence
NVIDIA documents Jetson PyTorch as JetPack-specific pip wheels with GPU
acceleration and cuDNN support, installed on top of the matching JetPack
version:
- https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html
- https://catalog.ngc.nvidia.com/orgs/nvidia/containers/l4t-pytorch
The NVIDIA NGC `l4t-pytorch` container is explicitly a Jetson/JetPack image and
requires matching the container tag to the installed JetPack/L4T release. The
`jetson-containers` project provides a practical composition path for PyTorch
and Transformers on Jetson, including compatible autotag selection:
- https://github.com/dusty-nv/jetson-containers
PyTorch upstream support is moving: PyTorch 2.11 release notes say PyPI wheels
now include CUDA 13.0 for Linux aarch64, while CUDA 12.6 and 12.8 wheels remain
available from `download.pytorch.org`. That improves generic ARM CUDA, but it
does not remove the need to match Jetson's installed JetPack/CUDA stack:
- https://github.com/pytorch/pytorch/releases
bitsandbytes currently publishes Linux aarch64 CUDA builds and lists targets
for CUDA 11.8 through 13.x. AGX Orin's Ampere GPU should satisfy the compute
capability requirement for 4-bit and 8-bit quantization, but the runtime still
has to be validated on the Jetson image actually used by contributors:
- https://huggingface.co/docs/bitsandbytes/en/installation
## Current repository gaps
- `pyproject.toml` maps Linux and Windows Torch resolution to the CPU-only
PyTorch index. That is correct for default CI, but it means the locked default
environment will not discover Jetson CUDA.
- `uv.lock` includes a Linux aarch64 bitsandbytes wheel, but Torch is locked as
`2.13.0+cpu` on Linux. A Jetson runtime must intentionally override Torch
after syncing the default lock, similar to the existing CUDA conditional lane.
- `.github/workflows/conditional-tests.yml` has one CUDA lane labeled
`self-hosted, linux, x64, cuda` and installs an official `cu130` Torch build.
It cannot run on Jetson because the runner labels and Torch source are x64
CUDA-specific.
- `docs/conditional-testing.md` documents generic CUDA and bitsandbytes probes,
not Jetson labels, JetPack requirements, or container startup.
- `Dockerfile` is based on `python:3.11-slim`, installs `requirements.txt`, and
is documented as local-only. It is not a Jetson image and should not be used
for a support claim.
- `obliteratus/device.py` can already detect CUDA through PyTorch, so no special
Jetson detection is needed for the first milestone. The blocker is installing
a CUDA-enabled Jetson PyTorch runtime and proving the existing device and
bitsandbytes contracts there.
- `tests/conditional/test_cuda_runtime.py` is a good starting probe, but it does
not record Jetson-specific platform evidence such as JetPack/L4T release,
CUDA version, Python version, architecture, Torch build, and bitsandbytes
binary availability.
## Implementation options
### Option A: Native Jetson runner, minimal repo changes
Attach a self-hosted runner on the Jetson with labels such as
`self-hosted`, `linux`, `ARM64`, `jetson`, and `cuda`. Add a new conditional
gate `jetson-cuda-runtime` that:
- syncs OBLITERATUS without replacing the system/vendor Jetson Torch build;
- verifies `platform.machine()` is `aarch64`;
- records `/etc/nv_tegra_release`, `nvcc --version`, `torch.__version__`,
`torch.version.cuda`, `torch.cuda.get_device_name(0)`, and
`bitsandbytes` import/runtime status;
- runs the existing CUDA and bitsandbytes probes.
Tradeoff: lowest abstraction and fastest to validate real hardware, but it
requires maintaining the Jetson's host Python/runtime state carefully.
### Option B: Jetson container lane
Base a Jetson-specific image on an NVIDIA L4T PyTorch image or build it with
`jetson-containers` from `pytorch` plus `transformers`. Install OBLITERATUS into
that image with dependency constraints that do not replace the container's
Torch. Run the same `jetson-cuda-runtime` probe inside the container.
Tradeoff: best reproducibility and closest to the reporter's concern about
patched/static libraries, but the image has to track JetPack/L4T tags and may
need per-JetPack dependency pins.
### Option C: Generic Linux ARM build plus separate Jetson UAT
Add a hosted or self-hosted Linux ARM packaging job that installs OBLITERATUS
with CPU Torch on aarch64, then keep Jetson CUDA as manual UAT evidence until a
stable runner exists.
Tradeoff: useful packaging signal for ARM contributors, but it must not be
presented as Jetson GPU support because it does not exercise CUDA, cuDNN,
device placement, quantization kernels, or JetPack compatibility.
## Recommended path
Use Option B for the support claim and Option C as a cheap early-warning signal.
The container path matches Jetson's dependency model, avoids polluting the
default lock with Jetson-only Torch URLs, and gives contributors a repeatable
recipe. A native runner can still be used to execute the container and collect
evidence. Treat generic ARM64 as Tier 0/Tier 1 preflight only; require
real-device Jetson smoke and release validation before publishing a support
claim.
## Acceptance criteria
- `docs/conditional-testing.md` documents Jetson as a separate conditional gate
with supported JetPack versions, runner labels, and local/container commands.
- `ci/conditional-test-policy.json` includes a `jetson-cuda-runtime` gate with a
tracking issue and evidence freshness rule.
- `.github/workflows/conditional-tests.yml` has a Jetson-selected job that runs
only on a labeled Jetson runner or through a Jetson-compatible container.
- The gate records JetPack/L4T, architecture, Torch CUDA version, CUDA device
name, memory, and bitsandbytes status in JSON evidence.
- The gate runs without skips:
`python scripts/run_conditional_gate.py jetson-cuda-runtime`.
- README support language says Jetson AGX Orin is supported only for the
specific JetPack/runtime combinations that have fresh green conditional
evidence.
## User guidance until support lands
Do not install from the default OBLITERATUS lock and expect Jetson CUDA to work.
Start from a JetPack-compatible PyTorch runtime, then install OBLITERATUS
without replacing Torch. If using a container, prefer an L4T PyTorch image or a
`jetson-containers` build that matches the device's JetPack/L4T release.
docs/platforms/jetson.md
+29 -17
View File
@@ -42,17 +42,19 @@ versions to NVIDIA framework containers/wheels and JetPack versions. A generic
ARM build can prove that Python code imports on `aarch64`; it cannot prove that
CUDA, cuDNN, TensorRT, or PyTorch CUDA dispatch works on Jetson.
## Initial support matrix
## Proposed initial support matrix
Start with the hardware reported in issue #31: Jetson AGX devices with 64 GB
unified memory.
Recommended first support tier:
This is a planning target, not a current support claim. The implementation must
replace the JetPack family with the exact patch installed on the available
runner before publishing compatibility:
| Tier | Hardware | JetPack | OS / CUDA baseline | Evidence requirement |
| --- | --- | --- | --- | --- |
| Target | Jetson AGX Orin 64 GB | 6.2 | Jetson Linux 36.4.3 / CUDA 12.6 | Native Jetson runner or NVIDIA Jetson container on Jetson hardware |
| Evaluate | Jetson AGX Thor | 7.x | Jetson Linux 38/39 / Ubuntu 24.04 / CUDA 13.x family | Separate runner and issue before claiming support |
| Initial candidate | Jetson AGX Orin 64 GB | 6.2.x, exact patch TBD | L4T/CUDA values from the selected patch | Native Jetson runner or pinned compatible container on Jetson hardware |
| Evaluate later | Jetson AGX Thor | 7.x, exact release TBD | Select only after NVIDIA's PyTorch compatibility table covers the release | Separate runner and evidence before claiming support |
| Legacy | Jetson AGX Xavier | 5.1.x | Jetson Linux 35.x / Ubuntu 20.04 / CUDA 11.x family | Defer unless a maintainer/user provides hardware and demand |
Do not collapse these tiers into one "ARM64" support claim.
@@ -72,7 +74,10 @@ runtime image. A Jetson runtime should use one of these approaches:
The lock policy should make the Jetson torch source explicit. The existing
Linux PR lock intentionally uses CPU-only PyTorch. A Jetson install path needs
an override or separate constraints file that preserves NVIDIA's Jetson PyTorch
runtime.
runtime. It must also prevent the generic PyPI Linux-aarch64 bitsandbytes wheel
from being selected: upstream documents that wheel as SBSA/server ARM and says
Jetson L4T/JetPack requires a source build. Until a pinned source build passes
on the selected device, bitsandbytes is unsupported for that tier.
## Conditional gate
@@ -85,6 +90,9 @@ Add a new gate instead of modifying the x64 CUDA gate:
- Evidence retention: same 30-day conditional-evidence policy as other hardware
gates
The job must run only from a trusted ref or reviewed maintainer dispatch. A
persistent self-hosted Jetson must never execute untrusted pull-request code.
The gate should verify:
- `platform.machine()` is `aarch64` or equivalent ARM64.
@@ -94,8 +102,8 @@ The gate should verify:
- OBLITERATUS resolves `device=auto` to `cuda`.
- A small CUDA tensor operation completes with finite output.
- The existing offloaded-surgery CUDA probe passes.
- `bitsandbytes` NF4/4-bit quantization is either proven on that exact Jetson
stack or documented as unsupported for the tier.
- A pinned, source-built `bitsandbytes` NF4/4-bit path is proven on that exact
Jetson stack, or bitsandbytes is documented as unsupported for the tier.
- A tiny Hugging Face model run passes only when the model-download gate is
explicitly selected and the runner has the required account/cache policy.
@@ -109,6 +117,9 @@ true:
wheel source.
- The Jetson conditional gate produces non-skipped green evidence on the exact
commit being claimed.
- The named `jetson-runtime` workflow job uses the documented runner labels and
retains `conditional-jetson-<run-attempt>` logs and environment metadata for
30 days.
- The release notes distinguish generic ARM importability from Jetson CUDA
support.
- The docs state memory expectations for 64 GB unified memory and recommend
@@ -121,16 +132,17 @@ and file cache. Treat "64 GB" as a capacity class, not guaranteed usable model
memory. Use small models for smoke tests, then move larger GPU validation to
dedicated CUDA hosts such as Titan when those resources are available.
Logging into a Hugging Face account is expected only for gated/private models,
license-gated models, or rate-limit avoidance. It should not be required for the
offline CPU PR gate or for the Jetson CUDA hardware probe. Any model-download
validation must remain an explicit conditional gate.
The Jetson hardware probe does not require a Hugging Face login. Model download
testing remains the separate, explicitly selected `model-download-runtime`
gate; credentials are relevant only when that selected model itself requires
them.
## Sources
- `REF-JETSON-PYTORCH-INSTALL`: NVIDIA, Installing PyTorch for Jetson Platform.
- `REF-JETSON-PYTORCH-RELEASES`: NVIDIA, PyTorch for Jetson Platform release notes.
- `REF-JETPACK-62`: NVIDIA, JetPack 6.2 release notes.
- `REF-JETPACK-7-DOWNLOADS`: NVIDIA, JetPack SDK downloads and notes.
- `REF-UV-PYTORCH`: Astral, Using uv with PyTorch.
- `REF-BITSANDBYTES-INSTALL`: Hugging Face, bitsandbytes installation guide.
- [NVIDIA: Installing PyTorch for Jetson Platform](https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html)
- [NVIDIA: PyTorch for Jetson compatibility table](https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform-release-notes/pytorch-jetson-rel.html)
- [NVIDIA: JetPack 6.2 release notes](https://docs.nvidia.com/jetson/archives/jetpack-archived/jetpack-62/release-notes/index.html)
- [NVIDIA: current JetPack downloads and notes](https://developer.nvidia.com/embedded/jetpack/downloads)
- [Astral: Using uv with PyTorch](https://docs.astral.sh/uv/guides/integration/pytorch/)
- [Hugging Face: bitsandbytes installation guide](https://huggingface.co/docs/bitsandbytes/installation)
- [GitHub: secure use of self-hosted runners](https://docs.github.com/en/actions/reference/security/secure-use#hardening-for-self-hosted-runners)