# Conditional test operations

The mandatory pull-request workflow remains CPU-only, offline, and credential-free.
Environment-bound contracts run through `Conditional tests` on manual dispatch,
weekly at 06:17 UTC Sunday, and for published releases. OBLITERATUS maintainers own
the workflow, policy, runner labels, credentials, and evidence review.

The canonical machine-readable policy is `ci/conditional-test-policy.json`. It maps
every environment-bound mature-coverage exclusion to a runnable gate, defines the
pinned tiny model, records prerequisites and expected cost, retains evidence for 30
days, and treats evidence older than eight days as stale. Every workflow run publishes
a summary showing selected, successful, failed, and not-selected/no-fresh-evidence
gates.

For a release, maintainers run the applicable hosted software gates manually against
the exact signed tag before publication. Publishing the release triggers the same
workflow again, and that release-event run is retained as the final freshness record.
Unselected hardware or credential-bound jobs remain explicit non-evidence. See
[`RELEASE_PROCESS.md`](RELEASE_PROCESS.md) for their place in the release sequence.

Unavailable hardware or credential-bound gates may use a committed environment
waiver for at most 30 days. Each waiver names one gate, a canonical tracking issue,
the reason, its opening and expiry dates, and the support claim it blocks. Invalid,
duplicate, future-dated, overlong, or expired waivers fail the policy job. The current
waivers are tracked by [issue #110](https://github.com/elder-plinius/OBLITERATUS/issues/110)
and expire on 2026-09-15; they are not evidence that any waived backend works.
They block support, compatibility, correctness, and performance claims for the
waived environment.

The current accelerator waivers cover missing GitHub runner registration, not
missing hardware. Titan has an RTX 4090 behind Gitea runners, and Mutsu is a 16 GB
Apple M4 builder. Exact-head operator probes may be attached to issue #110, but
they do not make the scheduled GitHub lanes runnable or establish a broad backend
support claim.

## Hosted gates

The model gate downloads only
`hf-internal-testing/tiny-random-gpt2@71034c5d8bde858ff824298bdedc65515b97d2b9`
with `trust_remote_code=False`. Its cache key includes the model revision, Python
version, and runner OS. The gate performs a forward pass, reopens the cache with Hub
and Transformers offline modes enabled, asserts an uncached model fails offline, and
runs the evaluator on two tiny samples. Its timeout is 25 minutes and its expected
download is below 100 MB.

The network-service gate uses a disposable loopback HTTP server and no credentials.
The operator-UI gate installs the locked `spaces` extra and constructs the Gradio
application without launching a listener. These gates cost less than ten hosted
runner-minutes each under normal conditions.

Run the same probes locally with:

```bash
uv sync --locked --extra dev
uv run --extra dev python scripts/run_conditional_gate.py model-download-runtime
uv run --extra dev python scripts/run_conditional_gate.py external-evaluation
uv run --extra dev python scripts/run_conditional_gate.py network-services

uv sync --locked --all-extras
uv run --all-extras python scripts/run_conditional_gate.py operator-ui
```

## CUDA and bitsandbytes

Attach a dedicated runner with the labels `self-hosted`, `linux`, `x64`, and `cuda`.
Set the repository variable `ENABLE_CUDA_GATE=true` for scheduled/release evidence,
or select CUDA during manual dispatch. The gate verifies CUDA discovery, automatic
selection, dtype selection, tensor placement, matrix multiplication, bitsandbytes
availability, NF4 quantization, dequantization, shape, placement, and finite output.
The expected cost is below 20 self-hosted runner-minutes. Mandatory Linux CI
deliberately locks CPU-only Torch. The selected CUDA job reads that exact locked
base version, replaces only Torch with the same-version official `cu130` build,
asserts a CUDA build was installed, and runs `uv pip check` before executing the
probes.

For an operator run on the labeled machine:

```bash
uv sync --locked --extra dev --extra quantization
CUDA_TORCH_VERSION="$(.venv/bin/python -c \
  'import torch; print(torch.__version__.split("+", 1)[0])')"
UV_TORCH_BACKEND=cu130 uv pip install --python .venv/bin/python \
  --reinstall-package torch "torch==$CUDA_TORCH_VERSION"
uv pip check --python .venv/bin/python
.venv/bin/python scripts/run_conditional_gate.py cuda-runtime
.venv/bin/python scripts/run_conditional_gate.py bitsandbytes-runtime
```

### Qwen3.5/Qwen3.8 hybrid runtime

The Qwen3.5/Qwen3.8 Gated DeltaNet path requires both FLA and causal-conv1d.
Install the optional extra only after selecting the CUDA build of PyTorch:

```bash
CUDA_HOME=/usr/local/cuda uv sync --locked --extra dev --extra qwen-hybrid
.venv/bin/python -c 'import torch, fla, causal_conv1d; print(torch.__version__)'
```

`causal-conv1d` wheels are keyed to the Python, PyTorch, CUDA, platform, and C++
ABI combination. Prefer an exact upstream wheel when one exists. A source build
must use a `CUDA_HOME` whose major version matches `torch.version.cuda`; the
system-default `nvcc` may point at a different toolkit. OBLITERATUS fails before
weight allocation if either extension cannot be imported.

The complete Qwen hybrid text model must fit one CUDA device with 15% free-memory
headroom. Generic `device_map="auto"` layer sharding is deliberately disabled for
this architecture because the recurrent-state execution path has not been
validated across devices.

Use the virtual environment interpreter directly after replacing Torch. A subsequent
`uv run` or `uv sync` without the CUDA override may restore the portable CPU wheel
from the lock. The agentic developer installer automates this ordering and performs
a real CUDA tensor probe; see `installer/setup.dev.manifest.yaml`.

## Quality-verifier coherence semantics

The built-in VERIFY stage does not treat length and vocabulary diversity as
coherence. Each fixed factual prompt has deterministic semantic anchors, and the
scorer rejects strong repetition and recognizable corpus contamination such as
scraped Q&A profile/answer fragments. This is intentionally local and deterministic:
release verification does not call another model or a network judge. Perplexity and
extended capability checks remain separate measurements.

On Linux workstations that also install cuDNN system-wide, the dynamic loader can
mix a host sublibrary with PyTorch's bundled cuDNN and raise
`CUDNN_STATUS_SUBLIBRARY_VERSION_MISMATCH`. The local launcher sets
`OBLITERATUS_DISABLE_CUDNN=1`, keeping causal convolutions and attention on
PyTorch's other CUDA kernels rather than moving the model to the CPU. Installer
verification runs actual causal `conv1d` and SDPA operations, not only a generic
CUDA allocation, so this failure is caught before a model pipeline begins.

Jetson CUDA support is tracked separately from this generic x64 CUDA lane. The
mandatory `Linux ARM64 preflight` uses GitHub's hosted `ubuntu-24.04-arm` runner
to prove locked CPU packaging, imports, CLI startup, and Jetson tooling contracts.
It is not GPU evidence. Jetson depends on a JetPack/L4T-matched CUDA, cuDNN, and
PyTorch runtime.

Physical testing uses only a trusted manual dispatch on the labels `self-hosted`,
`linux`, `ARM64`, and `jetson`. The job preserves NVIDIA's vendor PyTorch, runs
`jetson-runtime`, and uploads the sanitized `conditional-jetson-<run-attempt>`
artifact for 30 days. It never runs for a pull request, schedule, or release.
Contributor bootstrap, runner isolation, reporting commands, and acceptance
criteria are documented in the [NVIDIA Jetson support plan](platforms/jetson.md).

## Apple MPS and MLX

MPS uses a runner labeled `self-hosted`, `macOS`, `ARM64`, and `mps`; enable its
schedule with `ENABLE_MPS_GATE=true`. MLX uses the same first three labels plus `mlx`
and `ENABLE_MLX_GATE=true`. The MPS probe checks discovery, selection, dtype and
float64 fallback, placement, and a real matrix operation. The MLX probe uses the
locked `mlx==0.32.0` and `mlx-lm==0.31.3` packages and verifies imports, array
placement, evaluation, and matrix multiplication. Each should cost less than 20
self-hosted runner-minutes.

If the repository has no attached Apple runner, collect equivalent operator evidence
on Apple Silicon and attach the JSON and JUnit files to the tracking issue:

```bash
uv sync --locked --extra dev
uv run --extra dev python scripts/run_conditional_gate.py mps-runtime

uv sync --locked --extra dev --group mlx
uv run --extra dev --group mlx python scripts/run_conditional_gate.py mlx-runtime
```

## Remote provider

Remote evidence is opt-in. Configure a non-root, command-limited test account and:

- variables `OBLITERATUS_REMOTE_HOST`, `OBLITERATUS_REMOTE_USER`, and optionally
  `OBLITERATUS_REMOTE_PORT`;
- secrets `OBLITERATUS_REMOTE_KEY` and `OBLITERATUS_REMOTE_KNOWN_HOSTS`.

The known-hosts entry must be pinned after verifying the provider fingerprint through
an independent channel. The workflow writes credentials to mode-0600 temporary files,
never prints their contents, uses batch mode and strict host-key checking, and only
runs `echo ok` plus `python3 -c 'print(6 * 7)'`. Missing prerequisites produce an
actionable failure when the gate was selected; a direct local invocation may use
`--allow-missing` to record explicit `not_run` evidence. Expected cost is below five
hosted runner-minutes plus any provider charge.

## Result semantics

`scripts/run_conditional_gate.py` requires at least one executed test and rejects any
failure, error, or skip. A selected workflow job therefore cannot become green through
an availability skip or unconditional success conversion. The final summary also
fails if any selected job is not successful. Unselected jobs are explicitly reported
as `not_selected_no_fresh_evidence`; they are not evidence of backend support. An
unselected hardware or credential-bound gate with a valid waiver is reported as
`waived_no_support_claim`, including its tracker, expiry, and blocked claim. Selecting
and successfully running that gate produces `success` instead of relying on the
waiver.