feat: refresh model catalog and shared-host guidance

This commit is contained in:
Joseph Magly
2026-08-23 18:57:13 -04:00
parent bdcb5dd46f
commit cad100dd70
5 changed files with 302 additions and 2 deletions
+5
View File
@@ -162,6 +162,11 @@ obliteratus ui --auth user:pass # add basic auth
The `obliteratus ui` command adds a Rich terminal startup with GPU detection and hardware-appropriate model recommendations. You can also run `python app.py` directly (same thing the Space uses).
For a host shared by multiple people, do not treat one Gradio process as a
tenant boundary. Use the [shared GPU host deployment contract](docs/deployment/shared-gpu-host.md)
for per-user workspaces, credential isolation, GPU scheduling, storage, release,
and rollback requirements.
Install `.[spaces,quantization]` instead when the UI must load supported
bitsandbytes 8-bit or 4-bit models. Jetson users must follow the dedicated
[Jetson bootstrap](docs/platforms/jetson.md); its bitsandbytes path is not yet
+1
View File
@@ -287,6 +287,7 @@ _PROVIDER_NAMES = {
"openai-community": "OpenAI",
"openbmb": "OpenBMB",
"internlm": "Shanghai AI Lab",
"LiquidAI": "Liquid AI",
"stabilityai": "Stability AI",
"stepfun-ai": "StepFun",
"tiiuae": "TII (Falcon)",
+104
View File
@@ -0,0 +1,104 @@
# Shared GPU Host Deployment
This document defines the supported shape for preparing OBLITERATUS on a GPU
host used by more than one person. It is a deployment contract, not a claim
that the current Gradio process provides tenant isolation.
## Supported initial mode
Install one immutable, administrator-owned release. The preferred isolation is
per-user CLI execution, but a dedicated no-login service account may run one
explicitly shared trusted-team UI on loopback. Keep these boundaries separate:
| Boundary | Requirement |
|---|---|
| Release | Read-only to users; selected by an atomic `current` symlink |
| Workspace | One directory per Unix identity; mode `0700` |
| Model cache | Per-user by default; shared caches must be administrator-owned and read-only |
| Outputs | Per-user, quota-controlled, and excluded from backups unless explicitly promoted |
| Credentials | Per-user file, systemd credential, or trusted broker; never a shared environment file |
| GPU | Acquired through the host scheduler/lease broker before CUDA initialization |
| Network | CLI over SSH; any UI listens on loopback behind authenticated ingress |
Do not grant users write access to the release checkout, virtual environment,
service units, wrapper scripts, or another user's cache. Do not put users in the
Docker group merely to run OBLITERATUS; Docker group membership is effectively
root access.
## Gradio limitation
The current UI keeps loaded-model metadata and cleanup controls in process-global
state. A single shared process can therefore expose or remove another user's
session artifacts. Basic authentication controls entry but does not create
tenant isolation.
Until per-session artifact ownership, authorization, quotas, and cleanup tests
exist, use one of these modes:
1. shared CLI installation with private Unix workspaces (recommended);
2. one UI process per user, bound to loopback on a distinct port; or
3. a dedicated service account serving one trusted research team through one
explicitly shared workspace, bound to loopback.
Do not expose the UI directly on `0.0.0.0`. Gradio `--share` links are also not
an approved production ingress mechanism.
## GPU coordination contract
Static `nvidia-smi` checks do not prevent races. A host scheduler must reserve
the requested devices and VRAM before OBLITERATUS initializes CUDA, maintain the
lease for the entire process, and release it after CUDA allocations terminate.
Schedulers that drain another inference service commonly distinguish `acquire`
from `ready`. OBLITERATUS does not yet emit a portable post-model-load readiness
signal. Such deployments must not mark a lease ready immediately after process
start; add and test an application readiness hook first.
The requested VRAM must include weights, activation collection, verification,
checkpoint snapshots, CUDA context, and a safety margin. Multi-GPU sharding is
a capacity feature, not a general throughput multiplier.
## Filesystem layout
A conventional layout is:
```text
/srv/obliteratus/
├── releases/<commit>/ # administrator-owned source and .venv
├── current -> releases/... # atomic promotion target
├── users/<login>/ # mode 0700 workspaces and per-user caches
├── catalog/ # optional administrator-curated read-only models
└── staging/ # release verification before promotion
```
Large model checkpoints and rewritten outputs require a dedicated filesystem.
Set capacity and inode alerts, per-user quotas, and a retention policy before
allowing downloads. A safe default is no automatic backup of reproducible model
caches and outputs; back up only configs, reports, provenance, and explicitly
promoted research artifacts.
## Release and rollback
1. Select an immutable signed tag or reviewed commit.
2. Verify the source artifact and supply-chain evidence described in
[the release process](../RELEASE_PROCESS.md).
3. Build the locked environment in a new release directory.
4. Run `installer/scripts/verify.sh` on the target CUDA host.
5. Run a small, pinned-model smoke job through the real GPU scheduler.
6. Atomically repoint `current` only after all checks pass.
7. Retain the previous release until the observation window ends.
Rollback is an atomic symlink reversal followed by termination and restart of
new work. Do not change the release beneath a running job.
## Activation gate
Production activation requires all of the following:
- dedicated storage mounted and monitored;
- named users/groups and private workspace creation tested;
- GPU lease integration with a truthful post-load readiness signal;
- pinned release and locked CUDA environment verified;
- credential isolation and telemetry policy documented;
- smoke test, failure test, cancellation test, and rollback test passed;
- operator runbook and host inventory published outside the application repo.
+132 -1
View File
@@ -208,6 +208,15 @@ _PRESETS_LIST = [
recommended_dtype="bfloat16",
recommended_quantization="4bit",
),
ModelPreset(
name="Qwen3.8 2.4T-A95B",
hf_id="Qwen/Qwen3.8-2.4T-A95B",
description="Qwen3.8 flagship text model. 2.4T MoE (95B active), 1M context. Qwen license.",
tier="frontier",
params="2.4T MoE",
recommended_dtype="bfloat16",
recommended_quantization="4bit",
),
ModelPreset(
name="Qwen3.5-0.8B",
hf_id="Qwen/Qwen3.5-0.8B",
@@ -454,6 +463,24 @@ _PRESETS_LIST = [
recommended_dtype="bfloat16",
recommended_quantization="4bit",
),
ModelPreset(
name="DeepSeek-V4 Flash 0731",
hf_id="deepseek-ai/DeepSeek-V4-Flash-0731",
description="304B DeepSeek-V4 checkpoint with 1M context. MIT license.",
tier="frontier",
params="304B MoE",
recommended_dtype="bfloat16",
recommended_quantization="4bit",
),
ModelPreset(
name="DeepSeek-V4 Pro 0813",
hf_id="deepseek-ai/DeepSeek-V4-Pro-0813",
description="1.65T DeepSeek-V4 Pro checkpoint with 1M context. MIT license.",
tier="frontier",
params="1.65T MoE",
recommended_dtype="bfloat16",
recommended_quantization="4bit",
),
# ╔══════════════════════════════════════════════════════════════════╗
# ║ EleutherAI (Pythia) ║
@@ -616,6 +643,32 @@ _PRESETS_LIST = [
recommended_dtype="float16",
recommended_quantization="4bit",
),
ModelPreset(
name="Granite 4.1 3B",
hf_id="ibm-granite/granite-4.1-3b",
description="IBM Granite 4.1 3B instruct model. Apache 2.0.",
tier="small",
params="3.4B",
recommended_dtype="float16",
),
ModelPreset(
name="Granite 4.1 8B",
hf_id="ibm-granite/granite-4.1-8b",
description="IBM Granite 4.1 8B instruct model. Apache 2.0.",
tier="medium",
params="8.8B",
recommended_dtype="bfloat16",
recommended_quantization="4bit",
),
ModelPreset(
name="Granite 4.1 30B",
hf_id="ibm-granite/granite-4.1-30b",
description="IBM Granite 4.1 30B instruct model. Apache 2.0.",
tier="large",
params="28.9B",
recommended_dtype="bfloat16",
recommended_quantization="4bit",
),
# ╔══════════════════════════════════════════════════════════════════╗
# ║ Meta (LLaMA) ║
@@ -853,12 +906,21 @@ _PRESETS_LIST = [
ModelPreset(
name="GPT-OSS 20B",
hf_id="openai/gpt-oss-20b",
description="OpenAI's first open-weight MoE (20B total, 3.6B active). MIT license.",
description="OpenAI open-weight MoE (20B total, 3.6B active). Apache 2.0.",
tier="large",
params="20B MoE",
recommended_dtype="float16",
recommended_quantization="4bit",
),
ModelPreset(
name="GPT-OSS 120B",
hf_id="openai/gpt-oss-120b",
description="OpenAI open-weight MoE (120B total, 5.1B active). Apache 2.0.",
tier="frontier",
params="120B MoE",
recommended_dtype="bfloat16",
recommended_quantization="4bit",
),
# ╔══════════════════════════════════════════════════════════════════╗
# ║ OpenAI Community (GPT-2) ║
@@ -927,6 +989,51 @@ _PRESETS_LIST = [
recommended_quantization="4bit",
),
# ╔══════════════════════════════════════════════════════════════════╗
# ║ Liquid AI ║
# ╚══════════════════════════════════════════════════════════════════╝
ModelPreset(
name="LFM2.5 230M",
hf_id="LiquidAI/LFM2.5-230M",
description="Liquid AI's compact 230M LFM2.5 checkpoint. LFM Open License.",
tier="tiny",
params="230M",
recommended_dtype="float32",
),
ModelPreset(
name="LFM2.5 350M",
hf_id="LiquidAI/LFM2.5-350M",
description="Liquid AI's compact 350M LFM2.5 checkpoint. LFM Open License.",
tier="tiny",
params="354M",
recommended_dtype="float32",
),
ModelPreset(
name="LFM2.5 1.2B Instruct",
hf_id="LiquidAI/LFM2.5-1.2B-Instruct",
description="Liquid AI's 1.2B instruction-tuned LFM2.5. LFM Open License.",
tier="tiny",
params="1.17B",
recommended_dtype="float16",
),
ModelPreset(
name="LFM2.5 2.6B",
hf_id="LiquidAI/LFM2.5-2.6B",
description="Liquid AI's 2.6B LFM2.5 checkpoint. LFM Open License.",
tier="small",
params="2.7B",
recommended_dtype="float16",
),
ModelPreset(
name="LFM2.5 8B-A1B",
hf_id="LiquidAI/LFM2.5-8B-A1B",
description="Liquid AI LFM2.5 MoE (8.5B total, about 1B active). LFM Open License.",
tier="medium",
params="8.5B MoE",
recommended_dtype="bfloat16",
recommended_quantization="4bit",
),
# ╔══════════════════════════════════════════════════════════════════╗
# ║ Stability AI (StableLM) ║
# ╚══════════════════════════════════════════════════════════════════╝
@@ -1074,6 +1181,15 @@ _PRESETS_LIST = [
recommended_dtype="bfloat16",
recommended_quantization="4bit",
),
ModelPreset(
name="GLM-5.2",
hf_id="zai-org/GLM-5.2",
description="753B MoE successor to GLM-5 with 1M context. MIT license.",
tier="frontier",
params="753B MoE",
recommended_dtype="bfloat16",
recommended_quantization="4bit",
),
# ╔══════════════════════════════════════════════════════════════════╗
# ║ Community Fine-tunes: Uncensored / Abliterated ║
@@ -1160,6 +1276,21 @@ for p in _PRESETS_LIST:
MODEL_PRESETS[p.hf_id] = p
def refresh_presets_from_bestiary() -> int:
"""Merge configured BESTIARY open-weight records; curated records always win."""
from .bestiary_sync import extra_presets
added = extra_presets(ModelPreset, set(MODEL_PRESETS))
for preset in added:
MODEL_PRESETS[preset.hf_id] = preset
return len(added)
# This is a local/file lookup unless BESTIARY_CATALOG explicitly names a URL.
# An unavailable catalog is a safe no-op inside ``extra_presets``.
refresh_presets_from_bestiary()
def get_presets_by_tier(tier: str) -> list[ModelPreset]:
"""Return all presets for a compute tier."""
return [p for p in MODEL_PRESETS.values() if p.tier == tier]
+60 -1
View File
@@ -6,7 +6,7 @@ from dataclasses import dataclass
import pytest
from obliteratus import bestiary_sync, models_client
from obliteratus import bestiary_sync, models_client, presets
@dataclass
@@ -21,6 +21,28 @@ class Preset:
gated: bool
@pytest.mark.parametrize(
"model_id",
[
"Qwen/Qwen3.8-2.4T-A95B",
"deepseek-ai/DeepSeek-V4-Flash-0731",
"deepseek-ai/DeepSeek-V4-Pro-0813",
"ibm-granite/granite-4.1-3b",
"ibm-granite/granite-4.1-8b",
"ibm-granite/granite-4.1-30b",
"LiquidAI/LFM2.5-230M",
"LiquidAI/LFM2.5-350M",
"LiquidAI/LFM2.5-1.2B-Instruct",
"LiquidAI/LFM2.5-2.6B",
"LiquidAI/LFM2.5-8B-A1B",
"openai/gpt-oss-120b",
"zai-org/GLM-5.2",
],
)
def test_current_verified_models_are_curated(model_id: str) -> None:
assert model_id in presets.MODEL_PRESETS
@pytest.mark.parametrize(
"size, expected",
[
@@ -118,3 +140,40 @@ def test_preset_constructor_failure_does_not_return_partial_results(
super().__init__(**kwargs)
assert bestiary_sync.extra_presets(RejectSecond, set()) == []
def test_presets_refresh_merges_bestiary_records_without_replacing_curated(
monkeypatch: pytest.MonkeyPatch,
) -> None:
curated_id = next(iter(presets.MODEL_PRESETS))
curated = presets.MODEL_PRESETS[curated_id]
discovered = presets.ModelPreset(
name="Fresh 3B",
hf_id="example/Fresh-3B",
description="[BESTIARY · 2026-08-23] text.",
tier="small",
params="3B",
recommended_dtype="bfloat16",
)
replacement = presets.ModelPreset(
name="replacement",
hf_id=curated_id,
description="must not win",
tier="tiny",
params="1B",
recommended_dtype="float32",
)
seen_existing: set[str] = set()
def extras(_preset_type, existing: set[str]):
seen_existing.update(existing)
return [discovered] if curated_id in existing else [replacement, discovered]
monkeypatch.setattr(bestiary_sync, "extra_presets", extras)
try:
assert presets.refresh_presets_from_bestiary() == 1
assert curated_id in seen_existing
assert presets.MODEL_PRESETS[curated_id] is curated
assert presets.MODEL_PRESETS[discovered.hf_id] is discovered
finally:
presets.MODEL_PRESETS.pop(discovered.hf_id, None)