From 1e870ccee08a358058956f43e3546a8e403bea70 Mon Sep 17 00:00:00 2001 From: Joseph Magly <1159087+jmagly@users.noreply.github.com> Date: Fri, 21 Aug 2026 19:40:38 -0400 Subject: [PATCH] docs: harden Jetson runtime decision --- .../adr/ADR-001-jetson-runtime-support.md | 95 +++++++++++ .../jetson-support-architecture.md | 71 --------- .../REF-BITSANDBYTES-INSTALL-assessment.yaml | 5 +- .../REF-GITHUB-SELF-HOSTED-assessment.yaml | 13 ++ .../sources/REF-BITSANDBYTES-INSTALL.yaml | 3 +- .../sources/REF-GITHUB-SELF-HOSTED.yaml | 10 ++ .aiwg/testing/jetson-support-assessment.md | 24 ++- docs/jetson-support.md | 147 ------------------ docs/platforms/jetson.md | 46 ++++-- 9 files changed, 168 insertions(+), 246 deletions(-) create mode 100644 .aiwg/architecture/adr/ADR-001-jetson-runtime-support.md delete mode 100644 .aiwg/architecture/jetson-support-architecture.md create mode 100644 .aiwg/research/quality-assessments/REF-GITHUB-SELF-HOSTED-assessment.yaml create mode 100644 .aiwg/research/sources/REF-GITHUB-SELF-HOSTED.yaml delete mode 100644 docs/jetson-support.md diff --git a/.aiwg/architecture/adr/ADR-001-jetson-runtime-support.md b/.aiwg/architecture/adr/ADR-001-jetson-runtime-support.md new file mode 100644 index 0000000..5060166 --- /dev/null +++ b/.aiwg/architecture/adr/ADR-001-jetson-runtime-support.md @@ -0,0 +1,95 @@ +# ADR-001: Use a JetPack-pinned runtime for Jetson support + +Status: Proposed +Date: 2026-08-21 +Issue: https://github.com/elder-plinius/OBLITERATUS/issues/31 +Public plan: [docs/platforms/jetson.md](../../../docs/platforms/jetson.md) + +## Context + +OBLITERATUS currently resolves CPU-only PyTorch for Linux through its default +uv lock. Its hardware CUDA gate is explicitly x64 and replaces that locked +package with an upstream cu130 build. Application device discovery already +delegates to `torch.cuda.is_available()`. + +Jetson is therefore not primarily an application detection change. It is a +platform-runtime problem: the Jetson module, JetPack/L4T release, Python, +CUDA-family libraries, and PyTorch build or container must be compatible. A +generic ARM64 build can detect Python packaging defects but cannot establish +that this runtime works on Jetson hardware. + +## Options considered + +| Option | Benefit | Limitation | Decision | +| --- | --- | --- | --- | +| Generic ARM64 build only | Cheap packaging and CPU portability signal | Exercises no Jetson CUDA, L4T, or device memory behavior | Keep as preflight only | +| Native Jetson virtual environment | Direct and simple hardware validation | Host packages drift and the default lock can replace vendor PyTorch | Supported fallback | +| JetPack-compatible container on Jetson hardware | Reproducible runtime boundary and retained image provenance | Requires per-JetPack maintenance and a physical Jetson runner | Preferred | +| Build PyTorch and the complete CUDA stack from source | Maximum version control | High build cost and long-term platform maintenance burden | Reject for the initial tier | + +`jetson-containers` may accelerate prototyping, but it is a community project, +not the project's vendor trust anchor. A supported image must ultimately pin +audited base-image and package provenance. + +## Decision + +1. Keep the default PR environment and lock CPU-only. +2. Treat generic ARM64 CI as packaging preflight, never Jetson support evidence. +3. Define one exact initial matrix only after a runner is available: Jetson + module, JetPack patch, L4T, Python, PyTorch image or wheel, CUDA, cuDNN, + TensorRT, and bitsandbytes status. +4. Prefer an image derived from a JetPack-compatible NVIDIA PyTorch runtime and + run it on physical Jetson hardware. Preserve the image digest and binary + hashes in release evidence. +5. Add a separate `jetson-runtime` conditional gate. Its exact base labels are + `self-hosted`, `linux`, `ARM64`, and `jetson`; tier labels identify the module + and JetPack major version. +6. Run the hardware lane only by trusted manual, scheduled, or release + dispatch. Never execute untrusted pull-request code on the persistent + self-hosted Jetson runner. +7. Do not install the generic PyPI Linux-aarch64 bitsandbytes wheel on Jetson. + Upstream documents that wheel as SBSA/server ARM and requires a Jetson source + build. Quantization remains unsupported until a pinned source build passes + the exact hardware gate. + +## Consequences + +- Jetson support will have a narrower, explicit matrix rather than a broad ARM + claim. +- The implementation needs a Jetson-specific constraints/profile boundary so + `uv sync` cannot replace vendor PyTorch or select an incompatible + bitsandbytes wheel. +- Cross-building an ARM64 image may shorten image assembly, but only an + on-device run supplies CUDA support evidence. +- Vendor upgrades become release-affecting changes and must produce fresh gate + evidence. + +## Delivery and migration + +1. Inventory the available Jetson and record its exact JetPack/L4T stack. +2. Select and pin the initial Orin/JetPack 6.2.x matrix at an exact patch. +3. Add generic ARM64 package/import preflight without changing default PR + dependencies. +4. Add the Jetson constraints/profile and preferred container recipe. +5. Add the `jetson-runtime` policy entry, test probe, trusted workflow job, and + retained evidence artifact. +6. Validate CUDA discovery, device selection, a tiny CUDA operation, and the + offloaded-surgery probe. Validate source-built bitsandbytes separately. +7. Claim support only for matrices with fresh green evidence on the exact + release commit. + +## Rollback + +If the Jetson lane regresses, disable the affected matrix entry and remove its +support claim while retaining the generic ARM64 preflight. Revert the +Jetson-specific image/profile independently; do not change or weaken the +default CPU lock or x64 CUDA gate. Restore a matrix only after fresh physical +Jetson evidence passes. + +## Review record + +The 2026-08-21 review panel covered NVIDIA/platform compatibility, +build/maintainability, test/release evidence, and security/supply chain. It +required the Jetson-specific bitsandbytes source-build rule, physical-hardware +container evidence, one canonical public plan, exact gate naming, and an +untrusted-code restriction for the self-hosted runner. diff --git a/.aiwg/architecture/jetson-support-architecture.md b/.aiwg/architecture/jetson-support-architecture.md deleted file mode 100644 index c0433e5..0000000 --- a/.aiwg/architecture/jetson-support-architecture.md +++ /dev/null @@ -1,71 +0,0 @@ -# Jetson support architecture note - -Date: 2026-08-21 -Issue: https://github.com/elder-plinius/OBLITERATUS/issues/31 - -## Summary - -OBLITERATUS should support NVIDIA Jetson AGX as a JetPack-specific conditional -runtime target. A generic ARM build is useful as import/build coverage, but it -is not sufficient for Jetson CUDA support because the working runtime depends on -JetPack, L4T, CUDA, cuDNN, TensorRT, and NVIDIA's Jetson-compatible PyTorch -build or container. - -## Current repo state - -- `obliteratus/device.py` uses `torch.cuda.is_available()` for CUDA discovery. -- `docs/conditional-testing.md` defines hardware gates outside the mandatory - PR workflow. -- `.github/workflows/conditional-tests.yml` has an x64 CUDA gate using - `UV_TORCH_BACKEND=cu130`. -- `pyproject.toml` locks Linux PR resolution to CPU-only PyTorch through the - `pytorch-cpu` index. -- `Dockerfile` is a generic local `python:3.11-slim` image, not a Jetson image. - -These choices are correct for contributor PR turnaround, but they do not create -a Jetson-compatible runtime. - -## Required changes before claiming support - -1. Add a `jetson-runtime` conditional gate and policy entry. -2. Add a Jetson install path that preserves NVIDIA's Jetson PyTorch stack. -3. Add a Jetson container recipe or documented base-image override for the - selected JetPack tier. -4. Add a tiny hardware probe that records JetPack/L4T, CUDA, PyTorch, device - name, and OBLITERATUS `device=auto` behavior. -5. Decide whether `bitsandbytes` is supported, tier-limited, or disabled on the - selected Jetson stack based on hardware evidence. - -## Design constraints - -- Keep PR CI CPU-only and offline. -- Keep x64 CUDA and Jetson CUDA evidence separate. -- Do not let `uv sync` replace NVIDIA's Jetson PyTorch wheel/container runtime - with the generic CPU lock. -- Treat JetPack 6.x Orin and JetPack 7.x Thor/Orin as separate evidence tiers. -- Avoid publishing support claims without non-skipped conditional evidence. - -## Recommended first implementation - -Implement Jetson AGX Orin 64 GB on JetPack 6.2 first. Add a self-hosted runner -with labels `self-hosted`, `linux`, `ARM64`, `jetson`, `orin`, and -`jetpack-6`. The first gate should run only CUDA discovery, a small matrix -operation, OBLITERATUS device selection, and the existing offloaded-surgery -probe. Add `bitsandbytes` only after the same runner proves NF4 quantization -works with the selected PyTorch/CUDA stack. - -## Evidence basis - -- `REF-JETSON-PYTORCH-INSTALL` records NVIDIA's Jetson-specific PyTorch install - path and states the packages are intended for specified JetPack versions. -- `REF-JETSON-PYTORCH-RELEASES` maps PyTorch releases to NVIDIA framework - containers/wheels and JetPack versions. -- `REF-JETPACK-62` records JetPack 6.2 as Jetson Linux 36.4.3 and CUDA - 12.6. -- `REF-JETPACK-7-DOWNLOADS` records the current JetPack 7.2.1 stack as Jetson - Linux 39.2.1, Ubuntu 24.04, and CUDA 13.2.1. -- `REF-UV-PYTORCH` explains that PyTorch uses separate accelerator indexes and - local-version builds such as CPU and CUDA variants. -- `REF-BITSANDBYTES-INSTALL` documents Linux `aarch64` CUDA support targets, - but OBLITERATUS should still require project-specific Jetson evidence before - claiming quantization support. diff --git a/.aiwg/research/quality-assessments/REF-BITSANDBYTES-INSTALL-assessment.yaml b/.aiwg/research/quality-assessments/REF-BITSANDBYTES-INSTALL-assessment.yaml index 335284b..db71a09 100644 --- a/.aiwg/research/quality-assessments/REF-BITSANDBYTES-INSTALL-assessment.yaml +++ b/.aiwg/research/quality-assessments/REF-BITSANDBYTES-INSTALL-assessment.yaml @@ -6,8 +6,9 @@ upgrades: [] downgrades: - Library support matrix is not Jetson-specific OBLITERATUS runtime evidence. allowed_language: - - "bitsandbytes documents Linux aarch64 CUDA support..." - - "Quantization should still be proven on the selected Jetson stack..." + - "bitsandbytes documents generic Linux aarch64 CUDA wheels for SBSA/server ARM..." + - "Jetson requires a pinned source build that must be proven on the selected stack..." forbidden_language: - "bitsandbytes is confirmed for OBLITERATUS on Jetson..." - "Jetson quantization support can be claimed without hardware evidence..." + - "The generic Linux aarch64 wheel is compatible with Jetson..." diff --git a/.aiwg/research/quality-assessments/REF-GITHUB-SELF-HOSTED-assessment.yaml b/.aiwg/research/quality-assessments/REF-GITHUB-SELF-HOSTED-assessment.yaml new file mode 100644 index 0000000..9b05249 --- /dev/null +++ b/.aiwg/research/quality-assessments/REF-GITHUB-SELF-HOSTED-assessment.yaml @@ -0,0 +1,13 @@ +ref_id: REF-GITHUB-SELF-HOSTED +grade: MODERATE +baseline: LOW +source_type: platform_documentation +upgrades: + - Primary GitHub documentation for the runner trust boundary used by the proposed workflow. +downgrades: + - Documentation establishes platform risk, not proof of this repository's future runner configuration. +allowed_language: + - "GitHub warns that persistent self-hosted runners can be compromised by untrusted workflow code..." + - "The Jetson lane must be limited to trusted refs and maintainer-controlled dispatch..." +forbidden_language: + - "A runner label alone safely isolates untrusted pull requests..." diff --git a/.aiwg/research/sources/REF-BITSANDBYTES-INSTALL.yaml b/.aiwg/research/sources/REF-BITSANDBYTES-INSTALL.yaml index 258908e..2cf8a89 100644 --- a/.aiwg/research/sources/REF-BITSANDBYTES-INSTALL.yaml +++ b/.aiwg/research/sources/REF-BITSANDBYTES-INSTALL.yaml @@ -6,5 +6,6 @@ url: https://huggingface.co/docs/bitsandbytes/installation accessed_at: "2026-08-21T23:31:01Z" relevant_claims: - bitsandbytes supports NVIDIA CUDA GPUs with compute capability 6.0 or newer. - - Linux aarch64 CUDA builds are documented for CUDA Toolkit 11.8 through 13.2 with specific SM targets. + - Linux aarch64 CUDA wheels are documented for CUDA Toolkit 11.8 through 13.2 with specific SM targets. + - NVIDIA Jetson L4T/JetPack requires a source build; the published Linux aarch64 wheels target SBSA/server ARM and are not Jetson-compatible. - LLM.int8 requires Turing-class or newer hardware, while NF4/FP4 quantization requires Pascal-class or newer hardware. diff --git a/.aiwg/research/sources/REF-GITHUB-SELF-HOSTED.yaml b/.aiwg/research/sources/REF-GITHUB-SELF-HOSTED.yaml new file mode 100644 index 0000000..36027c2 --- /dev/null +++ b/.aiwg/research/sources/REF-GITHUB-SELF-HOSTED.yaml @@ -0,0 +1,10 @@ +id: REF-GITHUB-SELF-HOSTED +title: Self-hosted runners reference +source_type: platform_documentation +publisher: GitHub +url: https://docs.github.com/en/actions/reference/security/secure-use#hardening-for-self-hosted-runners +accessed_at: "2026-08-21T23:55:00Z" +relevant_claims: + - GitHub warns that self-hosted runners do not provide clean, ephemeral isolation for every job. + - GitHub recommends using self-hosted runners only with private repositories because repository forks can execute dangerous code through pull requests. + - A persistent Jetson runner must not execute untrusted pull-request code. diff --git a/.aiwg/testing/jetson-support-assessment.md b/.aiwg/testing/jetson-support-assessment.md index 685e2f5..71aef40 100644 --- a/.aiwg/testing/jetson-support-assessment.md +++ b/.aiwg/testing/jetson-support-assessment.md @@ -120,7 +120,8 @@ Purpose: prove the runtime actually works on a Jetson device. Environment: -- self-hosted Linux ARM64 Jetson runner; +- self-hosted Linux ARM64 Jetson runner with the exact labels `self-hosted`, + `linux`, `ARM64`, and `jetson`; - JetPack installed and active; - NVIDIA runtime / CUDA stack available. @@ -131,7 +132,8 @@ Checks: - a tiny tensor operation on GPU; - a minimal model-load or model-adapter smoke if the feature requires it; - a short CUDA sample or equivalent library smoke; -- optional containerized run using the JetPack container on the target. +- optional containerized run using the JetPack-compatible container on that + Jetson hardware. Acceptance: @@ -168,13 +170,13 @@ Acceptance: Use separate labels so the workflow can route cleanly and fail obviously if the hardware is absent. -- `self-hosted, linux, arm64, jetson` -- optional refinement: `self-hosted, linux, arm64, jetson, orin` -- optional refinement: `self-hosted, linux, arm64, jetpack-` +- `self-hosted, linux, ARM64, jetson` +- optional refinement: `self-hosted, linux, ARM64, jetson, orin` +- optional refinement: `self-hosted, linux, ARM64, jetson, jetpack-` Keep the generic ARM64 preflight on a separate label such as: -- `self-hosted, linux, arm64, arm64-preflight` +- `self-hosted, linux, ARM64, arm64-preflight` This avoids accidentally treating a generic ARM machine as a Jetson target. @@ -187,13 +189,15 @@ mechanism for a physical-device lane. Jetson support should be considered ready only when all of these are true: 1. The default PR gate still passes with no Jetson dependency. -2. A dedicated Jetson job passes on real hardware or an equivalent JetPack - target container. +2. A dedicated `jetson-runtime` job passes on physical Jetson hardware, either + natively or inside the pinned JetPack-compatible container. 3. The exact supported JetPack version is documented. 4. The exact NVIDIA wheel/container provenance is pinned and hashed. 5. The repo has a regression test that fails if Jetson detection or the CUDA smoke is broken. 6. The release notes state the supported JetPack / CUDA / TensorRT matrix. +7. The workflow retains the named `conditional-jetson-` log and + environment artifact for 30 days against the exact candidate commit. ## Memory and thermal constraints @@ -245,6 +249,10 @@ Controls to require: - record the exact JetPack release, CUDA version, and TensorRT version in the evidence trail; - treat any vendor wheel update as a release-affecting change. +- build bitsandbytes from a pinned source revision for Jetson, record the build + inputs and hash, and never substitute the generic SBSA Linux-aarch64 wheel; +- never execute untrusted pull-request code on the persistent self-hosted + Jetson runner. NVIDIA’s TensorRT documentation explicitly notes that JetPack deployments must remain on a TensorRT 10.x release supported by the JetPack version. That means diff --git a/docs/jetson-support.md b/docs/jetson-support.md deleted file mode 100644 index 8722a1b..0000000 --- a/docs/jetson-support.md +++ /dev/null @@ -1,147 +0,0 @@ -# NVIDIA Jetson support plan - -Issue: https://github.com/elder-plinius/OBLITERATUS/issues/31 - -Internal testing assessment: -[.aiwg/testing/jetson-support-assessment.md](../.aiwg/testing/jetson-support-assessment.md) - -## Decision - -OBLITERATUS should treat Jetson AGX support as a distinct platform target, not -as a generic Linux ARM build. Generic `linux/aarch64` packaging can prove that -the Python package resolves and imports on ARM, but it does not prove CUDA on -Jetson because Jetson devices depend on JetPack/L4T-matched CUDA, cuDNN, and -PyTorch builds. - -The first supported target should be Jetson AGX Orin 64 GB on JetPack 6.x, run -through either NVIDIA's Jetson PyTorch instructions or a JetPack-compatible -container. Xavier-era JetPack 5 can be documented as best-effort until a runner -exists. - -## Evidence - -NVIDIA documents Jetson PyTorch as JetPack-specific pip wheels with GPU -acceleration and cuDNN support, installed on top of the matching JetPack -version: - -- https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html -- https://catalog.ngc.nvidia.com/orgs/nvidia/containers/l4t-pytorch - -The NVIDIA NGC `l4t-pytorch` container is explicitly a Jetson/JetPack image and -requires matching the container tag to the installed JetPack/L4T release. The -`jetson-containers` project provides a practical composition path for PyTorch -and Transformers on Jetson, including compatible autotag selection: - -- https://github.com/dusty-nv/jetson-containers - -PyTorch upstream support is moving: PyTorch 2.11 release notes say PyPI wheels -now include CUDA 13.0 for Linux aarch64, while CUDA 12.6 and 12.8 wheels remain -available from `download.pytorch.org`. That improves generic ARM CUDA, but it -does not remove the need to match Jetson's installed JetPack/CUDA stack: - -- https://github.com/pytorch/pytorch/releases - -bitsandbytes currently publishes Linux aarch64 CUDA builds and lists targets -for CUDA 11.8 through 13.x. AGX Orin's Ampere GPU should satisfy the compute -capability requirement for 4-bit and 8-bit quantization, but the runtime still -has to be validated on the Jetson image actually used by contributors: - -- https://huggingface.co/docs/bitsandbytes/en/installation - -## Current repository gaps - -- `pyproject.toml` maps Linux and Windows Torch resolution to the CPU-only - PyTorch index. That is correct for default CI, but it means the locked default - environment will not discover Jetson CUDA. -- `uv.lock` includes a Linux aarch64 bitsandbytes wheel, but Torch is locked as - `2.13.0+cpu` on Linux. A Jetson runtime must intentionally override Torch - after syncing the default lock, similar to the existing CUDA conditional lane. -- `.github/workflows/conditional-tests.yml` has one CUDA lane labeled - `self-hosted, linux, x64, cuda` and installs an official `cu130` Torch build. - It cannot run on Jetson because the runner labels and Torch source are x64 - CUDA-specific. -- `docs/conditional-testing.md` documents generic CUDA and bitsandbytes probes, - not Jetson labels, JetPack requirements, or container startup. -- `Dockerfile` is based on `python:3.11-slim`, installs `requirements.txt`, and - is documented as local-only. It is not a Jetson image and should not be used - for a support claim. -- `obliteratus/device.py` can already detect CUDA through PyTorch, so no special - Jetson detection is needed for the first milestone. The blocker is installing - a CUDA-enabled Jetson PyTorch runtime and proving the existing device and - bitsandbytes contracts there. -- `tests/conditional/test_cuda_runtime.py` is a good starting probe, but it does - not record Jetson-specific platform evidence such as JetPack/L4T release, - CUDA version, Python version, architecture, Torch build, and bitsandbytes - binary availability. - -## Implementation options - -### Option A: Native Jetson runner, minimal repo changes - -Attach a self-hosted runner on the Jetson with labels such as -`self-hosted`, `linux`, `ARM64`, `jetson`, and `cuda`. Add a new conditional -gate `jetson-cuda-runtime` that: - -- syncs OBLITERATUS without replacing the system/vendor Jetson Torch build; -- verifies `platform.machine()` is `aarch64`; -- records `/etc/nv_tegra_release`, `nvcc --version`, `torch.__version__`, - `torch.version.cuda`, `torch.cuda.get_device_name(0)`, and - `bitsandbytes` import/runtime status; -- runs the existing CUDA and bitsandbytes probes. - -Tradeoff: lowest abstraction and fastest to validate real hardware, but it -requires maintaining the Jetson's host Python/runtime state carefully. - -### Option B: Jetson container lane - -Base a Jetson-specific image on an NVIDIA L4T PyTorch image or build it with -`jetson-containers` from `pytorch` plus `transformers`. Install OBLITERATUS into -that image with dependency constraints that do not replace the container's -Torch. Run the same `jetson-cuda-runtime` probe inside the container. - -Tradeoff: best reproducibility and closest to the reporter's concern about -patched/static libraries, but the image has to track JetPack/L4T tags and may -need per-JetPack dependency pins. - -### Option C: Generic Linux ARM build plus separate Jetson UAT - -Add a hosted or self-hosted Linux ARM packaging job that installs OBLITERATUS -with CPU Torch on aarch64, then keep Jetson CUDA as manual UAT evidence until a -stable runner exists. - -Tradeoff: useful packaging signal for ARM contributors, but it must not be -presented as Jetson GPU support because it does not exercise CUDA, cuDNN, -device placement, quantization kernels, or JetPack compatibility. - -## Recommended path - -Use Option B for the support claim and Option C as a cheap early-warning signal. -The container path matches Jetson's dependency model, avoids polluting the -default lock with Jetson-only Torch URLs, and gives contributors a repeatable -recipe. A native runner can still be used to execute the container and collect -evidence. Treat generic ARM64 as Tier 0/Tier 1 preflight only; require -real-device Jetson smoke and release validation before publishing a support -claim. - -## Acceptance criteria - -- `docs/conditional-testing.md` documents Jetson as a separate conditional gate - with supported JetPack versions, runner labels, and local/container commands. -- `ci/conditional-test-policy.json` includes a `jetson-cuda-runtime` gate with a - tracking issue and evidence freshness rule. -- `.github/workflows/conditional-tests.yml` has a Jetson-selected job that runs - only on a labeled Jetson runner or through a Jetson-compatible container. -- The gate records JetPack/L4T, architecture, Torch CUDA version, CUDA device - name, memory, and bitsandbytes status in JSON evidence. -- The gate runs without skips: - `python scripts/run_conditional_gate.py jetson-cuda-runtime`. -- README support language says Jetson AGX Orin is supported only for the - specific JetPack/runtime combinations that have fresh green conditional - evidence. - -## User guidance until support lands - -Do not install from the default OBLITERATUS lock and expect Jetson CUDA to work. -Start from a JetPack-compatible PyTorch runtime, then install OBLITERATUS -without replacing Torch. If using a container, prefer an L4T PyTorch image or a -`jetson-containers` build that matches the device's JetPack/L4T release. diff --git a/docs/platforms/jetson.md b/docs/platforms/jetson.md index e3edd7c..4be9af7 100644 --- a/docs/platforms/jetson.md +++ b/docs/platforms/jetson.md @@ -42,17 +42,19 @@ versions to NVIDIA framework containers/wheels and JetPack versions. A generic ARM build can prove that Python code imports on `aarch64`; it cannot prove that CUDA, cuDNN, TensorRT, or PyTorch CUDA dispatch works on Jetson. -## Initial support matrix +## Proposed initial support matrix Start with the hardware reported in issue #31: Jetson AGX devices with 64 GB unified memory. -Recommended first support tier: +This is a planning target, not a current support claim. The implementation must +replace the JetPack family with the exact patch installed on the available +runner before publishing compatibility: | Tier | Hardware | JetPack | OS / CUDA baseline | Evidence requirement | | --- | --- | --- | --- | --- | -| Target | Jetson AGX Orin 64 GB | 6.2 | Jetson Linux 36.4.3 / CUDA 12.6 | Native Jetson runner or NVIDIA Jetson container on Jetson hardware | -| Evaluate | Jetson AGX Thor | 7.x | Jetson Linux 38/39 / Ubuntu 24.04 / CUDA 13.x family | Separate runner and issue before claiming support | +| Initial candidate | Jetson AGX Orin 64 GB | 6.2.x, exact patch TBD | L4T/CUDA values from the selected patch | Native Jetson runner or pinned compatible container on Jetson hardware | +| Evaluate later | Jetson AGX Thor | 7.x, exact release TBD | Select only after NVIDIA's PyTorch compatibility table covers the release | Separate runner and evidence before claiming support | | Legacy | Jetson AGX Xavier | 5.1.x | Jetson Linux 35.x / Ubuntu 20.04 / CUDA 11.x family | Defer unless a maintainer/user provides hardware and demand | Do not collapse these tiers into one "ARM64" support claim. @@ -72,7 +74,10 @@ runtime image. A Jetson runtime should use one of these approaches: The lock policy should make the Jetson torch source explicit. The existing Linux PR lock intentionally uses CPU-only PyTorch. A Jetson install path needs an override or separate constraints file that preserves NVIDIA's Jetson PyTorch -runtime. +runtime. It must also prevent the generic PyPI Linux-aarch64 bitsandbytes wheel +from being selected: upstream documents that wheel as SBSA/server ARM and says +Jetson L4T/JetPack requires a source build. Until a pinned source build passes +on the selected device, bitsandbytes is unsupported for that tier. ## Conditional gate @@ -85,6 +90,9 @@ Add a new gate instead of modifying the x64 CUDA gate: - Evidence retention: same 30-day conditional-evidence policy as other hardware gates +The job must run only from a trusted ref or reviewed maintainer dispatch. A +persistent self-hosted Jetson must never execute untrusted pull-request code. + The gate should verify: - `platform.machine()` is `aarch64` or equivalent ARM64. @@ -94,8 +102,8 @@ The gate should verify: - OBLITERATUS resolves `device=auto` to `cuda`. - A small CUDA tensor operation completes with finite output. - The existing offloaded-surgery CUDA probe passes. -- `bitsandbytes` NF4/4-bit quantization is either proven on that exact Jetson - stack or documented as unsupported for the tier. +- A pinned, source-built `bitsandbytes` NF4/4-bit path is proven on that exact + Jetson stack, or bitsandbytes is documented as unsupported for the tier. - A tiny Hugging Face model run passes only when the model-download gate is explicitly selected and the runner has the required account/cache policy. @@ -109,6 +117,9 @@ true: wheel source. - The Jetson conditional gate produces non-skipped green evidence on the exact commit being claimed. +- The named `jetson-runtime` workflow job uses the documented runner labels and + retains `conditional-jetson-` logs and environment metadata for + 30 days. - The release notes distinguish generic ARM importability from Jetson CUDA support. - The docs state memory expectations for 64 GB unified memory and recommend @@ -121,16 +132,17 @@ and file cache. Treat "64 GB" as a capacity class, not guaranteed usable model memory. Use small models for smoke tests, then move larger GPU validation to dedicated CUDA hosts such as Titan when those resources are available. -Logging into a Hugging Face account is expected only for gated/private models, -license-gated models, or rate-limit avoidance. It should not be required for the -offline CPU PR gate or for the Jetson CUDA hardware probe. Any model-download -validation must remain an explicit conditional gate. +The Jetson hardware probe does not require a Hugging Face login. Model download +testing remains the separate, explicitly selected `model-download-runtime` +gate; credentials are relevant only when that selected model itself requires +them. ## Sources -- `REF-JETSON-PYTORCH-INSTALL`: NVIDIA, Installing PyTorch for Jetson Platform. -- `REF-JETSON-PYTORCH-RELEASES`: NVIDIA, PyTorch for Jetson Platform release notes. -- `REF-JETPACK-62`: NVIDIA, JetPack 6.2 release notes. -- `REF-JETPACK-7-DOWNLOADS`: NVIDIA, JetPack SDK downloads and notes. -- `REF-UV-PYTORCH`: Astral, Using uv with PyTorch. -- `REF-BITSANDBYTES-INSTALL`: Hugging Face, bitsandbytes installation guide. +- [NVIDIA: Installing PyTorch for Jetson Platform](https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html) +- [NVIDIA: PyTorch for Jetson compatibility table](https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform-release-notes/pytorch-jetson-rel.html) +- [NVIDIA: JetPack 6.2 release notes](https://docs.nvidia.com/jetson/archives/jetpack-archived/jetpack-62/release-notes/index.html) +- [NVIDIA: current JetPack downloads and notes](https://developer.nvidia.com/embedded/jetpack/downloads) +- [Astral: Using uv with PyTorch](https://docs.astral.sh/uv/guides/integration/pytorch/) +- [Hugging Face: bitsandbytes installation guide](https://huggingface.co/docs/bitsandbytes/installation) +- [GitHub: secure use of self-hosted runners](https://docs.github.com/en/actions/reference/security/secure-use#hardening-for-self-hosted-runners)