diff --git a/.aiwg/architecture/jetson-support-architecture.md b/.aiwg/architecture/jetson-support-architecture.md new file mode 100644 index 0000000..c0433e5 --- /dev/null +++ b/.aiwg/architecture/jetson-support-architecture.md @@ -0,0 +1,71 @@ +# Jetson support architecture note + +Date: 2026-08-21 +Issue: https://github.com/elder-plinius/OBLITERATUS/issues/31 + +## Summary + +OBLITERATUS should support NVIDIA Jetson AGX as a JetPack-specific conditional +runtime target. A generic ARM build is useful as import/build coverage, but it +is not sufficient for Jetson CUDA support because the working runtime depends on +JetPack, L4T, CUDA, cuDNN, TensorRT, and NVIDIA's Jetson-compatible PyTorch +build or container. + +## Current repo state + +- `obliteratus/device.py` uses `torch.cuda.is_available()` for CUDA discovery. +- `docs/conditional-testing.md` defines hardware gates outside the mandatory + PR workflow. +- `.github/workflows/conditional-tests.yml` has an x64 CUDA gate using + `UV_TORCH_BACKEND=cu130`. +- `pyproject.toml` locks Linux PR resolution to CPU-only PyTorch through the + `pytorch-cpu` index. +- `Dockerfile` is a generic local `python:3.11-slim` image, not a Jetson image. + +These choices are correct for contributor PR turnaround, but they do not create +a Jetson-compatible runtime. + +## Required changes before claiming support + +1. Add a `jetson-runtime` conditional gate and policy entry. +2. Add a Jetson install path that preserves NVIDIA's Jetson PyTorch stack. +3. Add a Jetson container recipe or documented base-image override for the + selected JetPack tier. +4. Add a tiny hardware probe that records JetPack/L4T, CUDA, PyTorch, device + name, and OBLITERATUS `device=auto` behavior. +5. Decide whether `bitsandbytes` is supported, tier-limited, or disabled on the + selected Jetson stack based on hardware evidence. + +## Design constraints + +- Keep PR CI CPU-only and offline. +- Keep x64 CUDA and Jetson CUDA evidence separate. +- Do not let `uv sync` replace NVIDIA's Jetson PyTorch wheel/container runtime + with the generic CPU lock. +- Treat JetPack 6.x Orin and JetPack 7.x Thor/Orin as separate evidence tiers. +- Avoid publishing support claims without non-skipped conditional evidence. + +## Recommended first implementation + +Implement Jetson AGX Orin 64 GB on JetPack 6.2 first. Add a self-hosted runner +with labels `self-hosted`, `linux`, `ARM64`, `jetson`, `orin`, and +`jetpack-6`. The first gate should run only CUDA discovery, a small matrix +operation, OBLITERATUS device selection, and the existing offloaded-surgery +probe. Add `bitsandbytes` only after the same runner proves NF4 quantization +works with the selected PyTorch/CUDA stack. + +## Evidence basis + +- `REF-JETSON-PYTORCH-INSTALL` records NVIDIA's Jetson-specific PyTorch install + path and states the packages are intended for specified JetPack versions. +- `REF-JETSON-PYTORCH-RELEASES` maps PyTorch releases to NVIDIA framework + containers/wheels and JetPack versions. +- `REF-JETPACK-62` records JetPack 6.2 as Jetson Linux 36.4.3 and CUDA + 12.6. +- `REF-JETPACK-7-DOWNLOADS` records the current JetPack 7.2.1 stack as Jetson + Linux 39.2.1, Ubuntu 24.04, and CUDA 13.2.1. +- `REF-UV-PYTORCH` explains that PyTorch uses separate accelerator indexes and + local-version builds such as CPU and CUDA variants. +- `REF-BITSANDBYTES-INSTALL` documents Linux `aarch64` CUDA support targets, + but OBLITERATUS should still require project-specific Jetson evidence before + claiming quantization support. diff --git a/.aiwg/research/quality-assessments/REF-BITSANDBYTES-INSTALL-assessment.yaml b/.aiwg/research/quality-assessments/REF-BITSANDBYTES-INSTALL-assessment.yaml new file mode 100644 index 0000000..335284b --- /dev/null +++ b/.aiwg/research/quality-assessments/REF-BITSANDBYTES-INSTALL-assessment.yaml @@ -0,0 +1,13 @@ +ref_id: REF-BITSANDBYTES-INSTALL +grade: LOW +baseline: LOW +source_type: library_documentation +upgrades: [] +downgrades: + - Library support matrix is not Jetson-specific OBLITERATUS runtime evidence. +allowed_language: + - "bitsandbytes documents Linux aarch64 CUDA support..." + - "Quantization should still be proven on the selected Jetson stack..." +forbidden_language: + - "bitsandbytes is confirmed for OBLITERATUS on Jetson..." + - "Jetson quantization support can be claimed without hardware evidence..." diff --git a/.aiwg/research/quality-assessments/REF-JETPACK-62-assessment.yaml b/.aiwg/research/quality-assessments/REF-JETPACK-62-assessment.yaml new file mode 100644 index 0000000..4928843 --- /dev/null +++ b/.aiwg/research/quality-assessments/REF-JETPACK-62-assessment.yaml @@ -0,0 +1,14 @@ +ref_id: REF-JETPACK-62 +grade: MODERATE +baseline: LOW +source_type: vendor_documentation +upgrades: + - Primary NVIDIA release notes for the recommended JetPack 6.2 target tier. +downgrades: + - Release notes establish platform stack contents, not OBLITERATUS runtime support. +allowed_language: + - "NVIDIA lists JetPack 6.2 as..." + - "JetPack 6.2 is an appropriate explicit platform tier..." +forbidden_language: + - "JetPack 6.2 support is certified..." + - "Generic ARM evidence proves this stack..." diff --git a/.aiwg/research/quality-assessments/REF-JETPACK-7-DOWNLOADS-assessment.yaml b/.aiwg/research/quality-assessments/REF-JETPACK-7-DOWNLOADS-assessment.yaml new file mode 100644 index 0000000..0cfa88a --- /dev/null +++ b/.aiwg/research/quality-assessments/REF-JETPACK-7-DOWNLOADS-assessment.yaml @@ -0,0 +1,13 @@ +ref_id: REF-JETPACK-7-DOWNLOADS +grade: MODERATE +baseline: LOW +source_type: vendor_documentation +upgrades: + - Current NVIDIA release/download page for JetPack 7 stack contents. +downgrades: + - Download page is not project runtime evidence. +allowed_language: + - "NVIDIA currently lists..." + - "JetPack 7 should be a separate evidence tier..." +forbidden_language: + - "JetPack 7 works with OBLITERATUS..." diff --git a/.aiwg/research/quality-assessments/REF-JETSON-PYTORCH-INSTALL-assessment.yaml b/.aiwg/research/quality-assessments/REF-JETSON-PYTORCH-INSTALL-assessment.yaml new file mode 100644 index 0000000..e388aef --- /dev/null +++ b/.aiwg/research/quality-assessments/REF-JETSON-PYTORCH-INSTALL-assessment.yaml @@ -0,0 +1,14 @@ +ref_id: REF-JETSON-PYTORCH-INSTALL +grade: MODERATE +baseline: LOW +source_type: vendor_documentation +upgrades: + - Primary vendor documentation for Jetson PyTorch installation. +downgrades: + - Vendor documentation can change without project-controlled reproducibility. +allowed_language: + - "NVIDIA documents..." + - "NVIDIA's install guide states..." +forbidden_language: + - "OBLITERATUS supports Jetson..." + - "This guarantees compatibility..." diff --git a/.aiwg/research/quality-assessments/REF-JETSON-PYTORCH-RELEASES-assessment.yaml b/.aiwg/research/quality-assessments/REF-JETSON-PYTORCH-RELEASES-assessment.yaml new file mode 100644 index 0000000..914dfb8 --- /dev/null +++ b/.aiwg/research/quality-assessments/REF-JETSON-PYTORCH-RELEASES-assessment.yaml @@ -0,0 +1,14 @@ +ref_id: REF-JETSON-PYTORCH-RELEASES +grade: MODERATE +baseline: LOW +source_type: vendor_documentation +upgrades: + - Primary vendor compatibility table for Jetson PyTorch releases. +downgrades: + - Release tables are necessary but not sufficient runtime evidence. +allowed_language: + - "NVIDIA maps..." + - "The compatibility table lists..." +forbidden_language: + - "All listed combinations work for OBLITERATUS..." + - "No hardware testing is needed..." diff --git a/.aiwg/research/quality-assessments/REF-UV-PYTORCH-assessment.yaml b/.aiwg/research/quality-assessments/REF-UV-PYTORCH-assessment.yaml new file mode 100644 index 0000000..9f2f867 --- /dev/null +++ b/.aiwg/research/quality-assessments/REF-UV-PYTORCH-assessment.yaml @@ -0,0 +1,13 @@ +ref_id: REF-UV-PYTORCH +grade: MODERATE +baseline: LOW +source_type: tool_documentation +upgrades: + - Primary tool documentation for uv PyTorch resolution behavior. +downgrades: + - Packaging behavior still needs validation in this repository's lock model. +allowed_language: + - "uv documents..." + - "PyTorch accelerator variants need explicit resolver handling..." +forbidden_language: + - "uv automatically solves Jetson packaging..." diff --git a/.aiwg/research/sources/REF-BITSANDBYTES-INSTALL.yaml b/.aiwg/research/sources/REF-BITSANDBYTES-INSTALL.yaml new file mode 100644 index 0000000..258908e --- /dev/null +++ b/.aiwg/research/sources/REF-BITSANDBYTES-INSTALL.yaml @@ -0,0 +1,10 @@ +id: REF-BITSANDBYTES-INSTALL +title: bitsandbytes Installation Guide +source_type: library_documentation +publisher: Hugging Face +url: https://huggingface.co/docs/bitsandbytes/installation +accessed_at: "2026-08-21T23:31:01Z" +relevant_claims: + - bitsandbytes supports NVIDIA CUDA GPUs with compute capability 6.0 or newer. + - Linux aarch64 CUDA builds are documented for CUDA Toolkit 11.8 through 13.2 with specific SM targets. + - LLM.int8 requires Turing-class or newer hardware, while NF4/FP4 quantization requires Pascal-class or newer hardware. diff --git a/.aiwg/research/sources/REF-JETPACK-62.yaml b/.aiwg/research/sources/REF-JETPACK-62.yaml new file mode 100644 index 0000000..d83b272 --- /dev/null +++ b/.aiwg/research/sources/REF-JETPACK-62.yaml @@ -0,0 +1,10 @@ +id: REF-JETPACK-62 +title: JetPack 6.2 Release Notes +source_type: vendor_documentation +publisher: NVIDIA +url: https://docs.nvidia.com/jetson/archives/jetpack-archived/jetpack-62/release-notes/index.html +accessed_at: "2026-08-21T23:31:01Z" +relevant_claims: + - JetPack 6.2 includes Jetson Linux 36.4.3. + - JetPack 6.2 includes a compute stack with CUDA 12.6, TensorRT 10.3, cuDNN 9.3, VPI 3.2, DLA 3.1, and DLFW 24.0. + - JetPack 6.2 targets Jetson Orin modules and includes updated Super Mode behavior for Orin Nano and Orin NX modules. diff --git a/.aiwg/research/sources/REF-JETPACK-7-DOWNLOADS.yaml b/.aiwg/research/sources/REF-JETPACK-7-DOWNLOADS.yaml new file mode 100644 index 0000000..8fdad1b --- /dev/null +++ b/.aiwg/research/sources/REF-JETPACK-7-DOWNLOADS.yaml @@ -0,0 +1,10 @@ +id: REF-JETPACK-7-DOWNLOADS +title: NVIDIA JetPack SDK Downloads and Notes +source_type: vendor_documentation +publisher: NVIDIA +url: https://developer.nvidia.com/embedded/jetpack/downloads +accessed_at: "2026-08-21T23:31:01Z" +relevant_claims: + - The current JetPack 7.2.1 release is paired with Jetson Linux 39.2.1. + - JetPack 7.2.1 lists CUDA 13.2.1, TensorRT 10.16.2, and cuDNN 9.20.0. + - JetPack 7 uses an Ubuntu 24.04 L4T base and aligns Jetson software with SBSA. diff --git a/.aiwg/research/sources/REF-JETSON-PYTORCH-INSTALL.yaml b/.aiwg/research/sources/REF-JETSON-PYTORCH-INSTALL.yaml new file mode 100644 index 0000000..674e4bc --- /dev/null +++ b/.aiwg/research/sources/REF-JETSON-PYTORCH-INSTALL.yaml @@ -0,0 +1,11 @@ +id: REF-JETSON-PYTORCH-INSTALL +title: Installing PyTorch for Jetson Platform +source_type: vendor_documentation +publisher: NVIDIA +url: https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html +accessed_at: "2026-08-21T23:31:01Z" +relevant_claims: + - NVIDIA provides Jetson PyTorch pip wheels with GPU acceleration and cuDNN support. + - The packages are intended to be installed on top of a specified JetPack version. + - Installation prerequisites include JetPack on the Jetson device and system packages. + - PyTorch installation verification starts by importing torch on the Jetson platform. diff --git a/.aiwg/research/sources/REF-JETSON-PYTORCH-RELEASES.yaml b/.aiwg/research/sources/REF-JETSON-PYTORCH-RELEASES.yaml new file mode 100644 index 0000000..25eb9d9 --- /dev/null +++ b/.aiwg/research/sources/REF-JETSON-PYTORCH-RELEASES.yaml @@ -0,0 +1,11 @@ +id: REF-JETSON-PYTORCH-RELEASES +title: PyTorch for Jetson Platform Release Notes +source_type: vendor_documentation +publisher: NVIDIA +url: https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform-release-notes/pytorch-jetson-rel.html +accessed_at: "2026-08-21T23:31:01Z" +relevant_claims: + - NVIDIA's compatibility table maps PyTorch versions to NVIDIA framework containers or wheels and JetPack versions. + - JetPack 6.2 entries map to NVIDIA framework containers 25.02 through 25.06. + - JetPack 7.x entries map to NVIDIA framework containers 25.08 and later. + - NVIDIA notes that standalone iGPU containers are no longer produced starting with the 26.03 release. diff --git a/.aiwg/research/sources/REF-UV-PYTORCH.yaml b/.aiwg/research/sources/REF-UV-PYTORCH.yaml new file mode 100644 index 0000000..5aaa54c --- /dev/null +++ b/.aiwg/research/sources/REF-UV-PYTORCH.yaml @@ -0,0 +1,11 @@ +id: REF-UV-PYTORCH +title: Using uv with PyTorch +source_type: tool_documentation +publisher: Astral +url: https://docs.astral.sh/uv/guides/integration/pytorch/ +accessed_at: "2026-08-21T23:31:01Z" +relevant_claims: + - uv can manage PyTorch dependencies while controlling accelerator selection. + - PyTorch wheels use dedicated indexes outside PyPI for many builds. + - PyTorch encodes accelerator builds in local version specifiers such as +cpu and +cu130. + - Different PyTorch accelerator builds are published on different indexes. diff --git a/docs/conditional-testing.md b/docs/conditional-testing.md index 10dc0ec..655e3da 100644 --- a/docs/conditional-testing.md +++ b/docs/conditional-testing.md @@ -84,7 +84,7 @@ Jetson CUDA support is tracked separately from this generic x64 CUDA lane. A generic Linux ARM build can prove package portability, but it does not prove Jetson GPU support because Jetson depends on a JetPack/L4T-matched CUDA, cuDNN, and PyTorch runtime. The support plan, recommended container path, and acceptance -criteria are documented in [NVIDIA Jetson support plan](jetson-support.md). +criteria are documented in [NVIDIA Jetson support plan](platforms/jetson.md). ## Apple MPS and MLX diff --git a/docs/platforms/jetson.md b/docs/platforms/jetson.md new file mode 100644 index 0000000..e3edd7c --- /dev/null +++ b/docs/platforms/jetson.md @@ -0,0 +1,136 @@ +# NVIDIA Jetson support plan + +Issue: https://github.com/elder-plinius/OBLITERATUS/issues/31 + +## Status + +Native Jetson AGX support is not claimed yet. OBLITERATUS should treat Jetson +as a dedicated conditional runtime lane, not as part of the default pull-request +gate and not as a generic Linux ARM build. + +The current OBLITERATUS CUDA path delegates discovery to PyTorch through +`torch.cuda.is_available()`. If a Jetson AGX host reports no CUDA inside +OBLITERATUS, the first thing to verify is the JetPack/L4T/PyTorch/container +stack, because NVIDIA publishes Jetson-specific PyTorch builds intended for +specified JetPack versions. + +## Decision + +Support Jetson through a JetPack-pinned runtime contract: + +- Keep ordinary PR CI CPU-only, offline, and architecture-neutral. +- Add a Jetson conditional gate once a Jetson runner is available. +- Prefer an NVIDIA-supported Jetson PyTorch container or NVIDIA Jetson PyTorch + wheel for the exact JetPack release under test. +- Do not use the existing x64 CUDA gate as Jetson evidence. +- Do not treat a generic `linux/arm64` build as evidence that CUDA works on + Jetson. + +## Why generic ARM is insufficient + +Jetson support couples at least five moving pieces: + +- Jetson hardware family and compute capability. +- JetPack version. +- Jetson Linux/L4T version and Ubuntu base image. +- CUDA, cuDNN, TensorRT, and related NVIDIA libraries. +- PyTorch build or container version. + +NVIDIA's Jetson PyTorch documentation says the PyTorch packages are installed +on top of a specified JetPack version, and the compatibility table maps PyTorch +versions to NVIDIA framework containers/wheels and JetPack versions. A generic +ARM build can prove that Python code imports on `aarch64`; it cannot prove that +CUDA, cuDNN, TensorRT, or PyTorch CUDA dispatch works on Jetson. + +## Initial support matrix + +Start with the hardware reported in issue #31: Jetson AGX devices with 64 GB +unified memory. + +Recommended first support tier: + +| Tier | Hardware | JetPack | OS / CUDA baseline | Evidence requirement | +| --- | --- | --- | --- | --- | +| Target | Jetson AGX Orin 64 GB | 6.2 | Jetson Linux 36.4.3 / CUDA 12.6 | Native Jetson runner or NVIDIA Jetson container on Jetson hardware | +| Evaluate | Jetson AGX Thor | 7.x | Jetson Linux 38/39 / Ubuntu 24.04 / CUDA 13.x family | Separate runner and issue before claiming support | +| Legacy | Jetson AGX Xavier | 5.1.x | Jetson Linux 35.x / Ubuntu 20.04 / CUDA 11.x family | Defer unless a maintainer/user provides hardware and demand | + +Do not collapse these tiers into one "ARM64" support claim. + +## Installation shape + +The generic local Dockerfile uses `python:3.11-slim` and is not the Jetson +runtime image. A Jetson runtime should use one of these approaches: + +1. Start from an NVIDIA Jetson-compatible PyTorch framework container for the + selected JetPack version, then install OBLITERATUS without replacing the + container's validated PyTorch stack. +2. On a flashed Jetson host, install the NVIDIA Jetson PyTorch wheel matching + the installed JetPack release, then install OBLITERATUS in a virtual + environment without allowing dependency resolution to replace `torch`. + +The lock policy should make the Jetson torch source explicit. The existing +Linux PR lock intentionally uses CPU-only PyTorch. A Jetson install path needs +an override or separate constraints file that preserves NVIDIA's Jetson PyTorch +runtime. + +## Conditional gate + +Add a new gate instead of modifying the x64 CUDA gate: + +- Gate id: `jetson-runtime` +- Runner labels: `self-hosted`, `linux`, `ARM64`, `jetson` +- Optional labels by tier: `orin`, `jetpack-6` or `thor`, `jetpack-7` +- Trigger: manual dispatch and release/scheduled validation only +- Evidence retention: same 30-day conditional-evidence policy as other hardware + gates + +The gate should verify: + +- `platform.machine()` is `aarch64` or equivalent ARM64. +- `torch.cuda.is_available()` is true. +- `torch.version.cuda` is not `None`. +- `torch.cuda.get_device_name(0)` identifies the Jetson GPU class. +- OBLITERATUS resolves `device=auto` to `cuda`. +- A small CUDA tensor operation completes with finite output. +- The existing offloaded-surgery CUDA probe passes. +- `bitsandbytes` NF4/4-bit quantization is either proven on that exact Jetson + stack or documented as unsupported for the tier. +- A tiny Hugging Face model run passes only when the model-download gate is + explicitly selected and the runner has the required account/cache policy. + +## Acceptance criteria + +Jetson support can be claimed for a tier only after all of the following are +true: + +- The supported Jetson module and JetPack version are named in this document. +- The install instructions pin the JetPack-compatible PyTorch container or + wheel source. +- The Jetson conditional gate produces non-skipped green evidence on the exact + commit being claimed. +- The release notes distinguish generic ARM importability from Jetson CUDA + support. +- The docs state memory expectations for 64 GB unified memory and recommend + small models for validation before large ablation runs. + +## Operational notes + +Jetson's unified memory is shared by the OS, CUDA, model weights, activations, +and file cache. Treat "64 GB" as a capacity class, not guaranteed usable model +memory. Use small models for smoke tests, then move larger GPU validation to +dedicated CUDA hosts such as Titan when those resources are available. + +Logging into a Hugging Face account is expected only for gated/private models, +license-gated models, or rate-limit avoidance. It should not be required for the +offline CPU PR gate or for the Jetson CUDA hardware probe. Any model-download +validation must remain an explicit conditional gate. + +## Sources + +- `REF-JETSON-PYTORCH-INSTALL`: NVIDIA, Installing PyTorch for Jetson Platform. +- `REF-JETSON-PYTORCH-RELEASES`: NVIDIA, PyTorch for Jetson Platform release notes. +- `REF-JETPACK-62`: NVIDIA, JetPack 6.2 release notes. +- `REF-JETPACK-7-DOWNLOADS`: NVIDIA, JetPack SDK downloads and notes. +- `REF-UV-PYTORCH`: Astral, Using uv with PyTorch. +- `REF-BITSANDBYTES-INSTALL`: Hugging Face, bitsandbytes installation guide.