diff --git a/.aiwg/testing/jetson-support-assessment.md b/.aiwg/testing/jetson-support-assessment.md new file mode 100644 index 0000000..685e2f5 --- /dev/null +++ b/.aiwg/testing/jetson-support-assessment.md @@ -0,0 +1,273 @@ +# Jetson support acceptance and release-validation strategy + +Date: 2026-08-21 +Scope: OBLITERATUS issue #31 +Status: assessment, not implementation + +## Executive conclusion + +Issue #31 cannot be validated or released on a generic ARM builder alone. +Generic ARM64 CI can cover repository-level portability, importability, +packaging, and CPU-only behavior, but Jetson support depends on the NVIDIA +JetPack software stack and on execution on Jetson-class hardware or a JetPack +container running on that hardware. + +The lowest-risk support model is: + +1. keep the default PR gate CPU-only and deterministic; +2. add a Jetson-specific conditional gate on a self-hosted ARM64 Jetson runner; +3. treat generic ARM64 CI as a preflight layer, not as acceptance evidence; +4. require real-device smoke evidence before declaring Jetson support. + +## What the current repo policy already says + +The project context already establishes that the default PR baseline is +CPU-safe, deterministic, and must not require accelerator, network, or remote +credentials. Conditional accelerator checks are explicitly outside the default +CPU job. + +The current conditional policy already has: + +- a CUDA gate on `self-hosted, linux, x64, cuda`; +- a documented waiver model for unavailable accelerator lanes; +- a GitHub Actions routing model that relies on runner labels and groups. + +That means Jetson support should be added as a new conditional lane, not folded +into the standard CPU PR job. + +## Evidence-based split: generic ARM64 vs Jetson device + +### Generic ARM64 builders can validate + +- Python packaging and metadata. +- Pure-Python imports and CLI/help entry points. +- CPU-only unit and boundary tests. +- Static/configuration logic that does not require CUDA or JetPack runtime + libraries. +- The fact that a workflow can target `ARM64` self-hosted runners. + +This is enough for portability regressions, but not enough for Jetson runtime +support. + +### Jetson hardware or JetPack container is required for + +- CUDA device discovery. +- CUDA execution and memory behavior. +- cuDNN / TensorRT / JetPack-specific library compatibility. +- Any claim that NVIDIA-provided Jetson PyTorch wheels run correctly. +- Any claim that the installed runtime actually sees a Jetson GPU. + +NVIDIA’s Jetson PyTorch installation guide says the provided wheels are meant +to be installed on top of a specified JetPack version on a Jetson device, and +verification is done by importing `torch` on the Jetson platform. + +NVIDIA’s Jetson Linux validation guide also says CUDA samples can be run +natively on the target or inside the JetPack container, which is the right +acceptance bar for hardware-backed validation. + +## Support tiers + +### Tier 0: source-only portability + +Purpose: prove the codebase does not contain obvious ARM-incompatible +assumptions. + +Environment: + +- GitHub-hosted Linux or self-hosted ARM64 runner; +- CPU-only; +- no NVIDIA driver, no JetPack, no CUDA. + +Checks: + +- `python -m build --sdist --wheel` +- install the built wheel into a clean venv +- `python -m obliteratus --help` +- import and module smoke tests +- all CPU-only pytest markers + +Acceptance: + +- no architecture-specific syntax/runtime breakage; +- no packaging or import regressions. + +### Tier 1: Jetson preflight on generic ARM64 + +Purpose: catch obvious Jetson-adjacent integration mistakes before touching +hardware. + +Environment: + +- ARM64 Linux runner without JetPack; +- optional cross-check against Jetson-targeted config files and dependency + pins. + +Checks: + +- validate Jetson-specific config manifests and workflow wiring; +- verify that Jetson jobs are gated behind dedicated labels and environment + variables; +- confirm that no Jetson-only dependency is pulled into the default PR job. + +Acceptance: + +- the repo can express a Jetson lane cleanly; +- default CI remains CPU-only. + +### Tier 2: Jetson hardware smoke + +Purpose: prove the runtime actually works on a Jetson device. + +Environment: + +- self-hosted Linux ARM64 Jetson runner; +- JetPack installed and active; +- NVIDIA runtime / CUDA stack available. + +Checks: + +- `torch` import from the JetPack-aligned wheel set; +- `torch.cuda.is_available()` and device enumeration; +- a tiny tensor operation on GPU; +- a minimal model-load or model-adapter smoke if the feature requires it; +- a short CUDA sample or equivalent library smoke; +- optional containerized run using the JetPack container on the target. + +Acceptance: + +- the device is recognized as CUDA-capable; +- the relevant Jetson wheel/container stack works on the intended JetPack + version; +- the workflow emits logs and artifacts that identify the exact JetPack / CUDA / + wheel set used. + +### Tier 3: release validation + +Purpose: certify a tagged release candidate. + +Environment: + +- Tier 2 Jetson hardware; +- pinned dependency set and reproducible artifacts; +- retained logs and hashes. + +Checks: + +- Tier 2 smoke; +- the repo’s full release-validation suite; +- supply-chain checks for vendor artifacts; +- any release note or compatibility matrix update. + +Acceptance: + +- tagged releases declare Jetson support only when Tier 2 and Tier 3 pass + against the exact tagged commit. + +## Recommended runner labels + +Use separate labels so the workflow can route cleanly and fail obviously if the +hardware is absent. + +- `self-hosted, linux, arm64, jetson` +- optional refinement: `self-hosted, linux, arm64, jetson, orin` +- optional refinement: `self-hosted, linux, arm64, jetpack-` + +Keep the generic ARM64 preflight on a separate label such as: + +- `self-hosted, linux, arm64, arm64-preflight` + +This avoids accidentally treating a generic ARM machine as a Jetson target. + +GitHub’s self-hosted runner routing is label-based, and jobs remain queued if +no matching runner is online. That makes an explicit Jetson label the right +mechanism for a physical-device lane. + +## Exact acceptance gates + +Jetson support should be considered ready only when all of these are true: + +1. The default PR gate still passes with no Jetson dependency. +2. A dedicated Jetson job passes on real hardware or an equivalent JetPack + target container. +3. The exact supported JetPack version is documented. +4. The exact NVIDIA wheel/container provenance is pinned and hashed. +5. The repo has a regression test that fails if Jetson detection or the CUDA + smoke is broken. +6. The release notes state the supported JetPack / CUDA / TensorRT matrix. + +## Memory and thermal constraints + +Jetson support should assume small-device variability, even on higher-memory +SKUs. + +Practical guardrails: + +- use tiny models and short sequences for smoke tests; +- cap wall time tightly; +- assert memory-sensitive paths with low-footprint fixtures; +- avoid benchmark-length runs on the Jetson lane; +- collect `nvidia-smi`-equivalent or device telemetry only if the platform + exposes it; +- do not rely on long repeated sweeps for acceptance. + +If a test is intended to validate memory pressure, it should do so with a +deterministic threshold and a short timeout, not with a human-observed +benchmark. + +## Rollback and waiver policy + +If Jetson hardware is unavailable, the repo should: + +- keep the Jetson lane optional and separately labeled; +- preserve the generic ARM64 preflight; +- publish a time-bounded waiver in the existing conditional-policy pattern; +- avoid claiming Jetson support in release notes. + +If a Jetson regression is found after support is published: + +- revert or patch the exact regression on the supported branch; +- keep the Jetson lane red until the hardware smoke passes again; +- do not widen the default PR gate to hide the failure. + +## Supply-chain and security considerations + +Jetson support increases supply-chain risk because the runtime depends on vendor +wheels and platform-specific system packages. + +Controls to require: + +- pin exact NVIDIA wheel URLs and versions; +- verify hashes for every downloaded binary artifact; +- prefer official JetPack / NVIDIA container images over ad hoc mixed library + stacks; +- do not mix arbitrary patched CUDA/TensorRT libraries into the supported + matrix; +- record the exact JetPack release, CUDA version, and TensorRT version in the + evidence trail; +- treat any vendor wheel update as a release-affecting change. + +NVIDIA’s TensorRT documentation explicitly notes that JetPack deployments must +remain on a TensorRT 10.x release supported by the JetPack version. That means +the Jetson compatibility matrix must be version-pinned, not “latest by default.” + +## Recommended implementation path + +1. Keep issue #31 open as a feature/specification item until the acceptance + matrix is merged. +2. Add a Jetson conditional policy entry and workflow job with an explicit + `jetson` runner label. +3. Add a minimal Jetson smoke test that checks import, CUDA discovery, and one + tiny GPU operation. +4. Add a generic ARM64 preflight job to catch packaging and configuration + regressions early. +5. Document the supported JetPack / CUDA / TensorRT matrix in the repo docs. +6. Promote Jetson claims only after a real-device run has passed on the exact + release candidate. + +## Source URLs + +- https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html +- https://docs.nvidia.com/jetson/archives/r36.5/DeveloperGuide/SD/TestPlanValidation.html +- https://docs.nvidia.com/deeplearning/tensorrt/latest/installing-tensorrt/installing.html +- https://docs.github.com/en/actions/reference/runners/self-hosted-runners +- https://docs.github.com/en/actions/reference/runners/github-hosted-runners diff --git a/docs/conditional-testing.md b/docs/conditional-testing.md index bc38504..10dc0ec 100644 --- a/docs/conditional-testing.md +++ b/docs/conditional-testing.md @@ -80,6 +80,12 @@ uv run --extra dev python scripts/run_conditional_gate.py cuda-runtime uv run --extra dev python scripts/run_conditional_gate.py bitsandbytes-runtime ``` +Jetson CUDA support is tracked separately from this generic x64 CUDA lane. A +generic Linux ARM build can prove package portability, but it does not prove +Jetson GPU support because Jetson depends on a JetPack/L4T-matched CUDA, cuDNN, +and PyTorch runtime. The support plan, recommended container path, and acceptance +criteria are documented in [NVIDIA Jetson support plan](jetson-support.md). + ## Apple MPS and MLX MPS uses a runner labeled `self-hosted`, `macOS`, `ARM64`, and `mps`; enable its diff --git a/docs/jetson-support.md b/docs/jetson-support.md new file mode 100644 index 0000000..8722a1b --- /dev/null +++ b/docs/jetson-support.md @@ -0,0 +1,147 @@ +# NVIDIA Jetson support plan + +Issue: https://github.com/elder-plinius/OBLITERATUS/issues/31 + +Internal testing assessment: +[.aiwg/testing/jetson-support-assessment.md](../.aiwg/testing/jetson-support-assessment.md) + +## Decision + +OBLITERATUS should treat Jetson AGX support as a distinct platform target, not +as a generic Linux ARM build. Generic `linux/aarch64` packaging can prove that +the Python package resolves and imports on ARM, but it does not prove CUDA on +Jetson because Jetson devices depend on JetPack/L4T-matched CUDA, cuDNN, and +PyTorch builds. + +The first supported target should be Jetson AGX Orin 64 GB on JetPack 6.x, run +through either NVIDIA's Jetson PyTorch instructions or a JetPack-compatible +container. Xavier-era JetPack 5 can be documented as best-effort until a runner +exists. + +## Evidence + +NVIDIA documents Jetson PyTorch as JetPack-specific pip wheels with GPU +acceleration and cuDNN support, installed on top of the matching JetPack +version: + +- https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html +- https://catalog.ngc.nvidia.com/orgs/nvidia/containers/l4t-pytorch + +The NVIDIA NGC `l4t-pytorch` container is explicitly a Jetson/JetPack image and +requires matching the container tag to the installed JetPack/L4T release. The +`jetson-containers` project provides a practical composition path for PyTorch +and Transformers on Jetson, including compatible autotag selection: + +- https://github.com/dusty-nv/jetson-containers + +PyTorch upstream support is moving: PyTorch 2.11 release notes say PyPI wheels +now include CUDA 13.0 for Linux aarch64, while CUDA 12.6 and 12.8 wheels remain +available from `download.pytorch.org`. That improves generic ARM CUDA, but it +does not remove the need to match Jetson's installed JetPack/CUDA stack: + +- https://github.com/pytorch/pytorch/releases + +bitsandbytes currently publishes Linux aarch64 CUDA builds and lists targets +for CUDA 11.8 through 13.x. AGX Orin's Ampere GPU should satisfy the compute +capability requirement for 4-bit and 8-bit quantization, but the runtime still +has to be validated on the Jetson image actually used by contributors: + +- https://huggingface.co/docs/bitsandbytes/en/installation + +## Current repository gaps + +- `pyproject.toml` maps Linux and Windows Torch resolution to the CPU-only + PyTorch index. That is correct for default CI, but it means the locked default + environment will not discover Jetson CUDA. +- `uv.lock` includes a Linux aarch64 bitsandbytes wheel, but Torch is locked as + `2.13.0+cpu` on Linux. A Jetson runtime must intentionally override Torch + after syncing the default lock, similar to the existing CUDA conditional lane. +- `.github/workflows/conditional-tests.yml` has one CUDA lane labeled + `self-hosted, linux, x64, cuda` and installs an official `cu130` Torch build. + It cannot run on Jetson because the runner labels and Torch source are x64 + CUDA-specific. +- `docs/conditional-testing.md` documents generic CUDA and bitsandbytes probes, + not Jetson labels, JetPack requirements, or container startup. +- `Dockerfile` is based on `python:3.11-slim`, installs `requirements.txt`, and + is documented as local-only. It is not a Jetson image and should not be used + for a support claim. +- `obliteratus/device.py` can already detect CUDA through PyTorch, so no special + Jetson detection is needed for the first milestone. The blocker is installing + a CUDA-enabled Jetson PyTorch runtime and proving the existing device and + bitsandbytes contracts there. +- `tests/conditional/test_cuda_runtime.py` is a good starting probe, but it does + not record Jetson-specific platform evidence such as JetPack/L4T release, + CUDA version, Python version, architecture, Torch build, and bitsandbytes + binary availability. + +## Implementation options + +### Option A: Native Jetson runner, minimal repo changes + +Attach a self-hosted runner on the Jetson with labels such as +`self-hosted`, `linux`, `ARM64`, `jetson`, and `cuda`. Add a new conditional +gate `jetson-cuda-runtime` that: + +- syncs OBLITERATUS without replacing the system/vendor Jetson Torch build; +- verifies `platform.machine()` is `aarch64`; +- records `/etc/nv_tegra_release`, `nvcc --version`, `torch.__version__`, + `torch.version.cuda`, `torch.cuda.get_device_name(0)`, and + `bitsandbytes` import/runtime status; +- runs the existing CUDA and bitsandbytes probes. + +Tradeoff: lowest abstraction and fastest to validate real hardware, but it +requires maintaining the Jetson's host Python/runtime state carefully. + +### Option B: Jetson container lane + +Base a Jetson-specific image on an NVIDIA L4T PyTorch image or build it with +`jetson-containers` from `pytorch` plus `transformers`. Install OBLITERATUS into +that image with dependency constraints that do not replace the container's +Torch. Run the same `jetson-cuda-runtime` probe inside the container. + +Tradeoff: best reproducibility and closest to the reporter's concern about +patched/static libraries, but the image has to track JetPack/L4T tags and may +need per-JetPack dependency pins. + +### Option C: Generic Linux ARM build plus separate Jetson UAT + +Add a hosted or self-hosted Linux ARM packaging job that installs OBLITERATUS +with CPU Torch on aarch64, then keep Jetson CUDA as manual UAT evidence until a +stable runner exists. + +Tradeoff: useful packaging signal for ARM contributors, but it must not be +presented as Jetson GPU support because it does not exercise CUDA, cuDNN, +device placement, quantization kernels, or JetPack compatibility. + +## Recommended path + +Use Option B for the support claim and Option C as a cheap early-warning signal. +The container path matches Jetson's dependency model, avoids polluting the +default lock with Jetson-only Torch URLs, and gives contributors a repeatable +recipe. A native runner can still be used to execute the container and collect +evidence. Treat generic ARM64 as Tier 0/Tier 1 preflight only; require +real-device Jetson smoke and release validation before publishing a support +claim. + +## Acceptance criteria + +- `docs/conditional-testing.md` documents Jetson as a separate conditional gate + with supported JetPack versions, runner labels, and local/container commands. +- `ci/conditional-test-policy.json` includes a `jetson-cuda-runtime` gate with a + tracking issue and evidence freshness rule. +- `.github/workflows/conditional-tests.yml` has a Jetson-selected job that runs + only on a labeled Jetson runner or through a Jetson-compatible container. +- The gate records JetPack/L4T, architecture, Torch CUDA version, CUDA device + name, memory, and bitsandbytes status in JSON evidence. +- The gate runs without skips: + `python scripts/run_conditional_gate.py jetson-cuda-runtime`. +- README support language says Jetson AGX Orin is supported only for the + specific JetPack/runtime combinations that have fresh green conditional + evidence. + +## User guidance until support lands + +Do not install from the default OBLITERATUS lock and expect Jetson CUDA to work. +Start from a JetPack-compatible PyTorch runtime, then install OBLITERATUS +without replacing Torch. If using a container, prefer an L4T PyTorch image or a +`jetson-containers` build that matches the device's JetPack/L4T release.