mirror of
https://github.com/elder-plinius/OBLITERATUS.git
synced 2026-08-29 22:20:36 +02:00
docs: harden Jetson runtime decision
This commit is contained in:
Joseph Magly
@@ -0,0 +1,95 @@
|
||||
# ADR-001: Use a JetPack-pinned runtime for Jetson support
|
||||
|
||||
Status: Proposed
|
||||
Date: 2026-08-21
|
||||
Issue: https://github.com/elder-plinius/OBLITERATUS/issues/31
|
||||
Public plan: [docs/platforms/jetson.md](../../../docs/platforms/jetson.md)
|
||||
|
||||
## Context
|
||||
|
||||
OBLITERATUS currently resolves CPU-only PyTorch for Linux through its default
|
||||
uv lock. Its hardware CUDA gate is explicitly x64 and replaces that locked
|
||||
package with an upstream cu130 build. Application device discovery already
|
||||
delegates to `torch.cuda.is_available()`.
|
||||
|
||||
Jetson is therefore not primarily an application detection change. It is a
|
||||
platform-runtime problem: the Jetson module, JetPack/L4T release, Python,
|
||||
CUDA-family libraries, and PyTorch build or container must be compatible. A
|
||||
generic ARM64 build can detect Python packaging defects but cannot establish
|
||||
that this runtime works on Jetson hardware.
|
||||
|
||||
## Options considered
|
||||
|
||||
| Option | Benefit | Limitation | Decision |
|
||||
| --- | --- | --- | --- |
|
||||
| Generic ARM64 build only | Cheap packaging and CPU portability signal | Exercises no Jetson CUDA, L4T, or device memory behavior | Keep as preflight only |
|
||||
| Native Jetson virtual environment | Direct and simple hardware validation | Host packages drift and the default lock can replace vendor PyTorch | Supported fallback |
|
||||
| JetPack-compatible container on Jetson hardware | Reproducible runtime boundary and retained image provenance | Requires per-JetPack maintenance and a physical Jetson runner | Preferred |
|
||||
| Build PyTorch and the complete CUDA stack from source | Maximum version control | High build cost and long-term platform maintenance burden | Reject for the initial tier |
|
||||
|
||||
`jetson-containers` may accelerate prototyping, but it is a community project,
|
||||
not the project's vendor trust anchor. A supported image must ultimately pin
|
||||
audited base-image and package provenance.
|
||||
|
||||
## Decision
|
||||
|
||||
1. Keep the default PR environment and lock CPU-only.
|
||||
2. Treat generic ARM64 CI as packaging preflight, never Jetson support evidence.
|
||||
3. Define one exact initial matrix only after a runner is available: Jetson
|
||||
module, JetPack patch, L4T, Python, PyTorch image or wheel, CUDA, cuDNN,
|
||||
TensorRT, and bitsandbytes status.
|
||||
4. Prefer an image derived from a JetPack-compatible NVIDIA PyTorch runtime and
|
||||
run it on physical Jetson hardware. Preserve the image digest and binary
|
||||
hashes in release evidence.
|
||||
5. Add a separate `jetson-runtime` conditional gate. Its exact base labels are
|
||||
`self-hosted`, `linux`, `ARM64`, and `jetson`; tier labels identify the module
|
||||
and JetPack major version.
|
||||
6. Run the hardware lane only by trusted manual, scheduled, or release
|
||||
dispatch. Never execute untrusted pull-request code on the persistent
|
||||
self-hosted Jetson runner.
|
||||
7. Do not install the generic PyPI Linux-aarch64 bitsandbytes wheel on Jetson.
|
||||
Upstream documents that wheel as SBSA/server ARM and requires a Jetson source
|
||||
build. Quantization remains unsupported until a pinned source build passes
|
||||
the exact hardware gate.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Jetson support will have a narrower, explicit matrix rather than a broad ARM
|
||||
claim.
|
||||
- The implementation needs a Jetson-specific constraints/profile boundary so
|
||||
`uv sync` cannot replace vendor PyTorch or select an incompatible
|
||||
bitsandbytes wheel.
|
||||
- Cross-building an ARM64 image may shorten image assembly, but only an
|
||||
on-device run supplies CUDA support evidence.
|
||||
- Vendor upgrades become release-affecting changes and must produce fresh gate
|
||||
evidence.
|
||||
|
||||
## Delivery and migration
|
||||
|
||||
1. Inventory the available Jetson and record its exact JetPack/L4T stack.
|
||||
2. Select and pin the initial Orin/JetPack 6.2.x matrix at an exact patch.
|
||||
3. Add generic ARM64 package/import preflight without changing default PR
|
||||
dependencies.
|
||||
4. Add the Jetson constraints/profile and preferred container recipe.
|
||||
5. Add the `jetson-runtime` policy entry, test probe, trusted workflow job, and
|
||||
retained evidence artifact.
|
||||
6. Validate CUDA discovery, device selection, a tiny CUDA operation, and the
|
||||
offloaded-surgery probe. Validate source-built bitsandbytes separately.
|
||||
7. Claim support only for matrices with fresh green evidence on the exact
|
||||
release commit.
|
||||
|
||||
## Rollback
|
||||
|
||||
If the Jetson lane regresses, disable the affected matrix entry and remove its
|
||||
support claim while retaining the generic ARM64 preflight. Revert the
|
||||
Jetson-specific image/profile independently; do not change or weaken the
|
||||
default CPU lock or x64 CUDA gate. Restore a matrix only after fresh physical
|
||||
Jetson evidence passes.
|
||||
|
||||
## Review record
|
||||
|
||||
The 2026-08-21 review panel covered NVIDIA/platform compatibility,
|
||||
build/maintainability, test/release evidence, and security/supply chain. It
|
||||
required the Jetson-specific bitsandbytes source-build rule, physical-hardware
|
||||
container evidence, one canonical public plan, exact gate naming, and an
|
||||
untrusted-code restriction for the self-hosted runner.
|
||||
@@ -1,71 +0,0 @@
|
||||
# Jetson support architecture note
|
||||
|
||||
Date: 2026-08-21
|
||||
Issue: https://github.com/elder-plinius/OBLITERATUS/issues/31
|
||||
|
||||
## Summary
|
||||
|
||||
OBLITERATUS should support NVIDIA Jetson AGX as a JetPack-specific conditional
|
||||
runtime target. A generic ARM build is useful as import/build coverage, but it
|
||||
is not sufficient for Jetson CUDA support because the working runtime depends on
|
||||
JetPack, L4T, CUDA, cuDNN, TensorRT, and NVIDIA's Jetson-compatible PyTorch
|
||||
build or container.
|
||||
|
||||
## Current repo state
|
||||
|
||||
- `obliteratus/device.py` uses `torch.cuda.is_available()` for CUDA discovery.
|
||||
- `docs/conditional-testing.md` defines hardware gates outside the mandatory
|
||||
PR workflow.
|
||||
- `.github/workflows/conditional-tests.yml` has an x64 CUDA gate using
|
||||
`UV_TORCH_BACKEND=cu130`.
|
||||
- `pyproject.toml` locks Linux PR resolution to CPU-only PyTorch through the
|
||||
`pytorch-cpu` index.
|
||||
- `Dockerfile` is a generic local `python:3.11-slim` image, not a Jetson image.
|
||||
|
||||
These choices are correct for contributor PR turnaround, but they do not create
|
||||
a Jetson-compatible runtime.
|
||||
|
||||
## Required changes before claiming support
|
||||
|
||||
1. Add a `jetson-runtime` conditional gate and policy entry.
|
||||
2. Add a Jetson install path that preserves NVIDIA's Jetson PyTorch stack.
|
||||
3. Add a Jetson container recipe or documented base-image override for the
|
||||
selected JetPack tier.
|
||||
4. Add a tiny hardware probe that records JetPack/L4T, CUDA, PyTorch, device
|
||||
name, and OBLITERATUS `device=auto` behavior.
|
||||
5. Decide whether `bitsandbytes` is supported, tier-limited, or disabled on the
|
||||
selected Jetson stack based on hardware evidence.
|
||||
|
||||
## Design constraints
|
||||
|
||||
- Keep PR CI CPU-only and offline.
|
||||
- Keep x64 CUDA and Jetson CUDA evidence separate.
|
||||
- Do not let `uv sync` replace NVIDIA's Jetson PyTorch wheel/container runtime
|
||||
with the generic CPU lock.
|
||||
- Treat JetPack 6.x Orin and JetPack 7.x Thor/Orin as separate evidence tiers.
|
||||
- Avoid publishing support claims without non-skipped conditional evidence.
|
||||
|
||||
## Recommended first implementation
|
||||
|
||||
Implement Jetson AGX Orin 64 GB on JetPack 6.2 first. Add a self-hosted runner
|
||||
with labels `self-hosted`, `linux`, `ARM64`, `jetson`, `orin`, and
|
||||
`jetpack-6`. The first gate should run only CUDA discovery, a small matrix
|
||||
operation, OBLITERATUS device selection, and the existing offloaded-surgery
|
||||
probe. Add `bitsandbytes` only after the same runner proves NF4 quantization
|
||||
works with the selected PyTorch/CUDA stack.
|
||||
|
||||
## Evidence basis
|
||||
|
||||
- `REF-JETSON-PYTORCH-INSTALL` records NVIDIA's Jetson-specific PyTorch install
|
||||
path and states the packages are intended for specified JetPack versions.
|
||||
- `REF-JETSON-PYTORCH-RELEASES` maps PyTorch releases to NVIDIA framework
|
||||
containers/wheels and JetPack versions.
|
||||
- `REF-JETPACK-62` records JetPack 6.2 as Jetson Linux 36.4.3 and CUDA
|
||||
12.6.
|
||||
- `REF-JETPACK-7-DOWNLOADS` records the current JetPack 7.2.1 stack as Jetson
|
||||
Linux 39.2.1, Ubuntu 24.04, and CUDA 13.2.1.
|
||||
- `REF-UV-PYTORCH` explains that PyTorch uses separate accelerator indexes and
|
||||
local-version builds such as CPU and CUDA variants.
|
||||
- `REF-BITSANDBYTES-INSTALL` documents Linux `aarch64` CUDA support targets,
|
||||
but OBLITERATUS should still require project-specific Jetson evidence before
|
||||
claiming quantization support.
|
||||
@@ -6,8 +6,9 @@ upgrades: []
|
||||
downgrades:
|
||||
- Library support matrix is not Jetson-specific OBLITERATUS runtime evidence.
|
||||
allowed_language:
|
||||
- "bitsandbytes documents Linux aarch64 CUDA support..."
|
||||
- "Quantization should still be proven on the selected Jetson stack..."
|
||||
- "bitsandbytes documents generic Linux aarch64 CUDA wheels for SBSA/server ARM..."
|
||||
- "Jetson requires a pinned source build that must be proven on the selected stack..."
|
||||
forbidden_language:
|
||||
- "bitsandbytes is confirmed for OBLITERATUS on Jetson..."
|
||||
- "Jetson quantization support can be claimed without hardware evidence..."
|
||||
- "The generic Linux aarch64 wheel is compatible with Jetson..."
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
ref_id: REF-GITHUB-SELF-HOSTED
|
||||
grade: MODERATE
|
||||
baseline: LOW
|
||||
source_type: platform_documentation
|
||||
upgrades:
|
||||
- Primary GitHub documentation for the runner trust boundary used by the proposed workflow.
|
||||
downgrades:
|
||||
- Documentation establishes platform risk, not proof of this repository's future runner configuration.
|
||||
allowed_language:
|
||||
- "GitHub warns that persistent self-hosted runners can be compromised by untrusted workflow code..."
|
||||
- "The Jetson lane must be limited to trusted refs and maintainer-controlled dispatch..."
|
||||
forbidden_language:
|
||||
- "A runner label alone safely isolates untrusted pull requests..."
|
||||
@@ -6,5 +6,6 @@ url: https://huggingface.co/docs/bitsandbytes/installation
|
||||
accessed_at: "2026-08-21T23:31:01Z"
|
||||
relevant_claims:
|
||||
- bitsandbytes supports NVIDIA CUDA GPUs with compute capability 6.0 or newer.
|
||||
- Linux aarch64 CUDA builds are documented for CUDA Toolkit 11.8 through 13.2 with specific SM targets.
|
||||
- Linux aarch64 CUDA wheels are documented for CUDA Toolkit 11.8 through 13.2 with specific SM targets.
|
||||
- NVIDIA Jetson L4T/JetPack requires a source build; the published Linux aarch64 wheels target SBSA/server ARM and are not Jetson-compatible.
|
||||
- LLM.int8 requires Turing-class or newer hardware, while NF4/FP4 quantization requires Pascal-class or newer hardware.
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
id: REF-GITHUB-SELF-HOSTED
|
||||
title: Self-hosted runners reference
|
||||
source_type: platform_documentation
|
||||
publisher: GitHub
|
||||
url: https://docs.github.com/en/actions/reference/security/secure-use#hardening-for-self-hosted-runners
|
||||
accessed_at: "2026-08-21T23:55:00Z"
|
||||
relevant_claims:
|
||||
- GitHub warns that self-hosted runners do not provide clean, ephemeral isolation for every job.
|
||||
- GitHub recommends using self-hosted runners only with private repositories because repository forks can execute dangerous code through pull requests.
|
||||
- A persistent Jetson runner must not execute untrusted pull-request code.
|
||||
@@ -120,7 +120,8 @@ Purpose: prove the runtime actually works on a Jetson device.
|
||||
|
||||
Environment:
|
||||
|
||||
- self-hosted Linux ARM64 Jetson runner;
|
||||
- self-hosted Linux ARM64 Jetson runner with the exact labels `self-hosted`,
|
||||
`linux`, `ARM64`, and `jetson`;
|
||||
- JetPack installed and active;
|
||||
- NVIDIA runtime / CUDA stack available.
|
||||
|
||||
@@ -131,7 +132,8 @@ Checks:
|
||||
- a tiny tensor operation on GPU;
|
||||
- a minimal model-load or model-adapter smoke if the feature requires it;
|
||||
- a short CUDA sample or equivalent library smoke;
|
||||
- optional containerized run using the JetPack container on the target.
|
||||
- optional containerized run using the JetPack-compatible container on that
|
||||
Jetson hardware.
|
||||
|
||||
Acceptance:
|
||||
|
||||
@@ -168,13 +170,13 @@ Acceptance:
|
||||
Use separate labels so the workflow can route cleanly and fail obviously if the
|
||||
hardware is absent.
|
||||
|
||||
- `self-hosted, linux, arm64, jetson`
|
||||
- optional refinement: `self-hosted, linux, arm64, jetson, orin`
|
||||
- optional refinement: `self-hosted, linux, arm64, jetpack-<major>`
|
||||
- `self-hosted, linux, ARM64, jetson`
|
||||
- optional refinement: `self-hosted, linux, ARM64, jetson, orin`
|
||||
- optional refinement: `self-hosted, linux, ARM64, jetson, jetpack-<major>`
|
||||
|
||||
Keep the generic ARM64 preflight on a separate label such as:
|
||||
|
||||
- `self-hosted, linux, arm64, arm64-preflight`
|
||||
- `self-hosted, linux, ARM64, arm64-preflight`
|
||||
|
||||
This avoids accidentally treating a generic ARM machine as a Jetson target.
|
||||
|
||||
@@ -187,13 +189,15 @@ mechanism for a physical-device lane.
|
||||
Jetson support should be considered ready only when all of these are true:
|
||||
|
||||
1. The default PR gate still passes with no Jetson dependency.
|
||||
2. A dedicated Jetson job passes on real hardware or an equivalent JetPack
|
||||
target container.
|
||||
2. A dedicated `jetson-runtime` job passes on physical Jetson hardware, either
|
||||
natively or inside the pinned JetPack-compatible container.
|
||||
3. The exact supported JetPack version is documented.
|
||||
4. The exact NVIDIA wheel/container provenance is pinned and hashed.
|
||||
5. The repo has a regression test that fails if Jetson detection or the CUDA
|
||||
smoke is broken.
|
||||
6. The release notes state the supported JetPack / CUDA / TensorRT matrix.
|
||||
7. The workflow retains the named `conditional-jetson-<run-attempt>` log and
|
||||
environment artifact for 30 days against the exact candidate commit.
|
||||
|
||||
## Memory and thermal constraints
|
||||
|
||||
@@ -245,6 +249,10 @@ Controls to require:
|
||||
- record the exact JetPack release, CUDA version, and TensorRT version in the
|
||||
evidence trail;
|
||||
- treat any vendor wheel update as a release-affecting change.
|
||||
- build bitsandbytes from a pinned source revision for Jetson, record the build
|
||||
inputs and hash, and never substitute the generic SBSA Linux-aarch64 wheel;
|
||||
- never execute untrusted pull-request code on the persistent self-hosted
|
||||
Jetson runner.
|
||||
|
||||
NVIDIA’s TensorRT documentation explicitly notes that JetPack deployments must
|
||||
remain on a TensorRT 10.x release supported by the JetPack version. That means
|
||||
|
||||
@@ -1,147 +0,0 @@
|
||||
# NVIDIA Jetson support plan
|
||||
|
||||
Issue: https://github.com/elder-plinius/OBLITERATUS/issues/31
|
||||
|
||||
Internal testing assessment:
|
||||
[.aiwg/testing/jetson-support-assessment.md](../.aiwg/testing/jetson-support-assessment.md)
|
||||
|
||||
## Decision
|
||||
|
||||
OBLITERATUS should treat Jetson AGX support as a distinct platform target, not
|
||||
as a generic Linux ARM build. Generic `linux/aarch64` packaging can prove that
|
||||
the Python package resolves and imports on ARM, but it does not prove CUDA on
|
||||
Jetson because Jetson devices depend on JetPack/L4T-matched CUDA, cuDNN, and
|
||||
PyTorch builds.
|
||||
|
||||
The first supported target should be Jetson AGX Orin 64 GB on JetPack 6.x, run
|
||||
through either NVIDIA's Jetson PyTorch instructions or a JetPack-compatible
|
||||
container. Xavier-era JetPack 5 can be documented as best-effort until a runner
|
||||
exists.
|
||||
|
||||
## Evidence
|
||||
|
||||
NVIDIA documents Jetson PyTorch as JetPack-specific pip wheels with GPU
|
||||
acceleration and cuDNN support, installed on top of the matching JetPack
|
||||
version:
|
||||
|
||||
- https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html
|
||||
- https://catalog.ngc.nvidia.com/orgs/nvidia/containers/l4t-pytorch
|
||||
|
||||
The NVIDIA NGC `l4t-pytorch` container is explicitly a Jetson/JetPack image and
|
||||
requires matching the container tag to the installed JetPack/L4T release. The
|
||||
`jetson-containers` project provides a practical composition path for PyTorch
|
||||
and Transformers on Jetson, including compatible autotag selection:
|
||||
|
||||
- https://github.com/dusty-nv/jetson-containers
|
||||
|
||||
PyTorch upstream support is moving: PyTorch 2.11 release notes say PyPI wheels
|
||||
now include CUDA 13.0 for Linux aarch64, while CUDA 12.6 and 12.8 wheels remain
|
||||
available from `download.pytorch.org`. That improves generic ARM CUDA, but it
|
||||
does not remove the need to match Jetson's installed JetPack/CUDA stack:
|
||||
|
||||
- https://github.com/pytorch/pytorch/releases
|
||||
|
||||
bitsandbytes currently publishes Linux aarch64 CUDA builds and lists targets
|
||||
for CUDA 11.8 through 13.x. AGX Orin's Ampere GPU should satisfy the compute
|
||||
capability requirement for 4-bit and 8-bit quantization, but the runtime still
|
||||
has to be validated on the Jetson image actually used by contributors:
|
||||
|
||||
- https://huggingface.co/docs/bitsandbytes/en/installation
|
||||
|
||||
## Current repository gaps
|
||||
|
||||
- `pyproject.toml` maps Linux and Windows Torch resolution to the CPU-only
|
||||
PyTorch index. That is correct for default CI, but it means the locked default
|
||||
environment will not discover Jetson CUDA.
|
||||
- `uv.lock` includes a Linux aarch64 bitsandbytes wheel, but Torch is locked as
|
||||
`2.13.0+cpu` on Linux. A Jetson runtime must intentionally override Torch
|
||||
after syncing the default lock, similar to the existing CUDA conditional lane.
|
||||
- `.github/workflows/conditional-tests.yml` has one CUDA lane labeled
|
||||
`self-hosted, linux, x64, cuda` and installs an official `cu130` Torch build.
|
||||
It cannot run on Jetson because the runner labels and Torch source are x64
|
||||
CUDA-specific.
|
||||
- `docs/conditional-testing.md` documents generic CUDA and bitsandbytes probes,
|
||||
not Jetson labels, JetPack requirements, or container startup.
|
||||
- `Dockerfile` is based on `python:3.11-slim`, installs `requirements.txt`, and
|
||||
is documented as local-only. It is not a Jetson image and should not be used
|
||||
for a support claim.
|
||||
- `obliteratus/device.py` can already detect CUDA through PyTorch, so no special
|
||||
Jetson detection is needed for the first milestone. The blocker is installing
|
||||
a CUDA-enabled Jetson PyTorch runtime and proving the existing device and
|
||||
bitsandbytes contracts there.
|
||||
- `tests/conditional/test_cuda_runtime.py` is a good starting probe, but it does
|
||||
not record Jetson-specific platform evidence such as JetPack/L4T release,
|
||||
CUDA version, Python version, architecture, Torch build, and bitsandbytes
|
||||
binary availability.
|
||||
|
||||
## Implementation options
|
||||
|
||||
### Option A: Native Jetson runner, minimal repo changes
|
||||
|
||||
Attach a self-hosted runner on the Jetson with labels such as
|
||||
`self-hosted`, `linux`, `ARM64`, `jetson`, and `cuda`. Add a new conditional
|
||||
gate `jetson-cuda-runtime` that:
|
||||
|
||||
- syncs OBLITERATUS without replacing the system/vendor Jetson Torch build;
|
||||
- verifies `platform.machine()` is `aarch64`;
|
||||
- records `/etc/nv_tegra_release`, `nvcc --version`, `torch.__version__`,
|
||||
`torch.version.cuda`, `torch.cuda.get_device_name(0)`, and
|
||||
`bitsandbytes` import/runtime status;
|
||||
- runs the existing CUDA and bitsandbytes probes.
|
||||
|
||||
Tradeoff: lowest abstraction and fastest to validate real hardware, but it
|
||||
requires maintaining the Jetson's host Python/runtime state carefully.
|
||||
|
||||
### Option B: Jetson container lane
|
||||
|
||||
Base a Jetson-specific image on an NVIDIA L4T PyTorch image or build it with
|
||||
`jetson-containers` from `pytorch` plus `transformers`. Install OBLITERATUS into
|
||||
that image with dependency constraints that do not replace the container's
|
||||
Torch. Run the same `jetson-cuda-runtime` probe inside the container.
|
||||
|
||||
Tradeoff: best reproducibility and closest to the reporter's concern about
|
||||
patched/static libraries, but the image has to track JetPack/L4T tags and may
|
||||
need per-JetPack dependency pins.
|
||||
|
||||
### Option C: Generic Linux ARM build plus separate Jetson UAT
|
||||
|
||||
Add a hosted or self-hosted Linux ARM packaging job that installs OBLITERATUS
|
||||
with CPU Torch on aarch64, then keep Jetson CUDA as manual UAT evidence until a
|
||||
stable runner exists.
|
||||
|
||||
Tradeoff: useful packaging signal for ARM contributors, but it must not be
|
||||
presented as Jetson GPU support because it does not exercise CUDA, cuDNN,
|
||||
device placement, quantization kernels, or JetPack compatibility.
|
||||
|
||||
## Recommended path
|
||||
|
||||
Use Option B for the support claim and Option C as a cheap early-warning signal.
|
||||
The container path matches Jetson's dependency model, avoids polluting the
|
||||
default lock with Jetson-only Torch URLs, and gives contributors a repeatable
|
||||
recipe. A native runner can still be used to execute the container and collect
|
||||
evidence. Treat generic ARM64 as Tier 0/Tier 1 preflight only; require
|
||||
real-device Jetson smoke and release validation before publishing a support
|
||||
claim.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- `docs/conditional-testing.md` documents Jetson as a separate conditional gate
|
||||
with supported JetPack versions, runner labels, and local/container commands.
|
||||
- `ci/conditional-test-policy.json` includes a `jetson-cuda-runtime` gate with a
|
||||
tracking issue and evidence freshness rule.
|
||||
- `.github/workflows/conditional-tests.yml` has a Jetson-selected job that runs
|
||||
only on a labeled Jetson runner or through a Jetson-compatible container.
|
||||
- The gate records JetPack/L4T, architecture, Torch CUDA version, CUDA device
|
||||
name, memory, and bitsandbytes status in JSON evidence.
|
||||
- The gate runs without skips:
|
||||
`python scripts/run_conditional_gate.py jetson-cuda-runtime`.
|
||||
- README support language says Jetson AGX Orin is supported only for the
|
||||
specific JetPack/runtime combinations that have fresh green conditional
|
||||
evidence.
|
||||
|
||||
## User guidance until support lands
|
||||
|
||||
Do not install from the default OBLITERATUS lock and expect Jetson CUDA to work.
|
||||
Start from a JetPack-compatible PyTorch runtime, then install OBLITERATUS
|
||||
without replacing Torch. If using a container, prefer an L4T PyTorch image or a
|
||||
`jetson-containers` build that matches the device's JetPack/L4T release.
|
||||
+29
-17
@@ -42,17 +42,19 @@ versions to NVIDIA framework containers/wheels and JetPack versions. A generic
|
||||
ARM build can prove that Python code imports on `aarch64`; it cannot prove that
|
||||
CUDA, cuDNN, TensorRT, or PyTorch CUDA dispatch works on Jetson.
|
||||
|
||||
## Initial support matrix
|
||||
## Proposed initial support matrix
|
||||
|
||||
Start with the hardware reported in issue #31: Jetson AGX devices with 64 GB
|
||||
unified memory.
|
||||
|
||||
Recommended first support tier:
|
||||
This is a planning target, not a current support claim. The implementation must
|
||||
replace the JetPack family with the exact patch installed on the available
|
||||
runner before publishing compatibility:
|
||||
|
||||
| Tier | Hardware | JetPack | OS / CUDA baseline | Evidence requirement |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Target | Jetson AGX Orin 64 GB | 6.2 | Jetson Linux 36.4.3 / CUDA 12.6 | Native Jetson runner or NVIDIA Jetson container on Jetson hardware |
|
||||
| Evaluate | Jetson AGX Thor | 7.x | Jetson Linux 38/39 / Ubuntu 24.04 / CUDA 13.x family | Separate runner and issue before claiming support |
|
||||
| Initial candidate | Jetson AGX Orin 64 GB | 6.2.x, exact patch TBD | L4T/CUDA values from the selected patch | Native Jetson runner or pinned compatible container on Jetson hardware |
|
||||
| Evaluate later | Jetson AGX Thor | 7.x, exact release TBD | Select only after NVIDIA's PyTorch compatibility table covers the release | Separate runner and evidence before claiming support |
|
||||
| Legacy | Jetson AGX Xavier | 5.1.x | Jetson Linux 35.x / Ubuntu 20.04 / CUDA 11.x family | Defer unless a maintainer/user provides hardware and demand |
|
||||
|
||||
Do not collapse these tiers into one "ARM64" support claim.
|
||||
@@ -72,7 +74,10 @@ runtime image. A Jetson runtime should use one of these approaches:
|
||||
The lock policy should make the Jetson torch source explicit. The existing
|
||||
Linux PR lock intentionally uses CPU-only PyTorch. A Jetson install path needs
|
||||
an override or separate constraints file that preserves NVIDIA's Jetson PyTorch
|
||||
runtime.
|
||||
runtime. It must also prevent the generic PyPI Linux-aarch64 bitsandbytes wheel
|
||||
from being selected: upstream documents that wheel as SBSA/server ARM and says
|
||||
Jetson L4T/JetPack requires a source build. Until a pinned source build passes
|
||||
on the selected device, bitsandbytes is unsupported for that tier.
|
||||
|
||||
## Conditional gate
|
||||
|
||||
@@ -85,6 +90,9 @@ Add a new gate instead of modifying the x64 CUDA gate:
|
||||
- Evidence retention: same 30-day conditional-evidence policy as other hardware
|
||||
gates
|
||||
|
||||
The job must run only from a trusted ref or reviewed maintainer dispatch. A
|
||||
persistent self-hosted Jetson must never execute untrusted pull-request code.
|
||||
|
||||
The gate should verify:
|
||||
|
||||
- `platform.machine()` is `aarch64` or equivalent ARM64.
|
||||
@@ -94,8 +102,8 @@ The gate should verify:
|
||||
- OBLITERATUS resolves `device=auto` to `cuda`.
|
||||
- A small CUDA tensor operation completes with finite output.
|
||||
- The existing offloaded-surgery CUDA probe passes.
|
||||
- `bitsandbytes` NF4/4-bit quantization is either proven on that exact Jetson
|
||||
stack or documented as unsupported for the tier.
|
||||
- A pinned, source-built `bitsandbytes` NF4/4-bit path is proven on that exact
|
||||
Jetson stack, or bitsandbytes is documented as unsupported for the tier.
|
||||
- A tiny Hugging Face model run passes only when the model-download gate is
|
||||
explicitly selected and the runner has the required account/cache policy.
|
||||
|
||||
@@ -109,6 +117,9 @@ true:
|
||||
wheel source.
|
||||
- The Jetson conditional gate produces non-skipped green evidence on the exact
|
||||
commit being claimed.
|
||||
- The named `jetson-runtime` workflow job uses the documented runner labels and
|
||||
retains `conditional-jetson-<run-attempt>` logs and environment metadata for
|
||||
30 days.
|
||||
- The release notes distinguish generic ARM importability from Jetson CUDA
|
||||
support.
|
||||
- The docs state memory expectations for 64 GB unified memory and recommend
|
||||
@@ -121,16 +132,17 @@ and file cache. Treat "64 GB" as a capacity class, not guaranteed usable model
|
||||
memory. Use small models for smoke tests, then move larger GPU validation to
|
||||
dedicated CUDA hosts such as Titan when those resources are available.
|
||||
|
||||
Logging into a Hugging Face account is expected only for gated/private models,
|
||||
license-gated models, or rate-limit avoidance. It should not be required for the
|
||||
offline CPU PR gate or for the Jetson CUDA hardware probe. Any model-download
|
||||
validation must remain an explicit conditional gate.
|
||||
The Jetson hardware probe does not require a Hugging Face login. Model download
|
||||
testing remains the separate, explicitly selected `model-download-runtime`
|
||||
gate; credentials are relevant only when that selected model itself requires
|
||||
them.
|
||||
|
||||
## Sources
|
||||
|
||||
- `REF-JETSON-PYTORCH-INSTALL`: NVIDIA, Installing PyTorch for Jetson Platform.
|
||||
- `REF-JETSON-PYTORCH-RELEASES`: NVIDIA, PyTorch for Jetson Platform release notes.
|
||||
- `REF-JETPACK-62`: NVIDIA, JetPack 6.2 release notes.
|
||||
- `REF-JETPACK-7-DOWNLOADS`: NVIDIA, JetPack SDK downloads and notes.
|
||||
- `REF-UV-PYTORCH`: Astral, Using uv with PyTorch.
|
||||
- `REF-BITSANDBYTES-INSTALL`: Hugging Face, bitsandbytes installation guide.
|
||||
- [NVIDIA: Installing PyTorch for Jetson Platform](https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html)
|
||||
- [NVIDIA: PyTorch for Jetson compatibility table](https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform-release-notes/pytorch-jetson-rel.html)
|
||||
- [NVIDIA: JetPack 6.2 release notes](https://docs.nvidia.com/jetson/archives/jetpack-archived/jetpack-62/release-notes/index.html)
|
||||
- [NVIDIA: current JetPack downloads and notes](https://developer.nvidia.com/embedded/jetpack/downloads)
|
||||
- [Astral: Using uv with PyTorch](https://docs.astral.sh/uv/guides/integration/pytorch/)
|
||||
- [Hugging Face: bitsandbytes installation guide](https://huggingface.co/docs/bitsandbytes/installation)
|
||||
- [GitHub: secure use of self-hosted runners](https://docs.github.com/en/actions/reference/security/secure-use#hardening-for-self-hosted-runners)
|
||||
|
||||
Reference in New Issue
Block a user