mirror of
https://github.com/elder-plinius/OBLITERATUS.git
synced 2026-08-30 06:30:37 +02:00
feat: refresh model catalog and shared-host guidance
This commit is contained in:
@@ -0,0 +1,104 @@
|
||||
# Shared GPU Host Deployment
|
||||
|
||||
This document defines the supported shape for preparing OBLITERATUS on a GPU
|
||||
host used by more than one person. It is a deployment contract, not a claim
|
||||
that the current Gradio process provides tenant isolation.
|
||||
|
||||
## Supported initial mode
|
||||
|
||||
Install one immutable, administrator-owned release. The preferred isolation is
|
||||
per-user CLI execution, but a dedicated no-login service account may run one
|
||||
explicitly shared trusted-team UI on loopback. Keep these boundaries separate:
|
||||
|
||||
| Boundary | Requirement |
|
||||
|---|---|
|
||||
| Release | Read-only to users; selected by an atomic `current` symlink |
|
||||
| Workspace | One directory per Unix identity; mode `0700` |
|
||||
| Model cache | Per-user by default; shared caches must be administrator-owned and read-only |
|
||||
| Outputs | Per-user, quota-controlled, and excluded from backups unless explicitly promoted |
|
||||
| Credentials | Per-user file, systemd credential, or trusted broker; never a shared environment file |
|
||||
| GPU | Acquired through the host scheduler/lease broker before CUDA initialization |
|
||||
| Network | CLI over SSH; any UI listens on loopback behind authenticated ingress |
|
||||
|
||||
Do not grant users write access to the release checkout, virtual environment,
|
||||
service units, wrapper scripts, or another user's cache. Do not put users in the
|
||||
Docker group merely to run OBLITERATUS; Docker group membership is effectively
|
||||
root access.
|
||||
|
||||
## Gradio limitation
|
||||
|
||||
The current UI keeps loaded-model metadata and cleanup controls in process-global
|
||||
state. A single shared process can therefore expose or remove another user's
|
||||
session artifacts. Basic authentication controls entry but does not create
|
||||
tenant isolation.
|
||||
|
||||
Until per-session artifact ownership, authorization, quotas, and cleanup tests
|
||||
exist, use one of these modes:
|
||||
|
||||
1. shared CLI installation with private Unix workspaces (recommended);
|
||||
2. one UI process per user, bound to loopback on a distinct port; or
|
||||
3. a dedicated service account serving one trusted research team through one
|
||||
explicitly shared workspace, bound to loopback.
|
||||
|
||||
Do not expose the UI directly on `0.0.0.0`. Gradio `--share` links are also not
|
||||
an approved production ingress mechanism.
|
||||
|
||||
## GPU coordination contract
|
||||
|
||||
Static `nvidia-smi` checks do not prevent races. A host scheduler must reserve
|
||||
the requested devices and VRAM before OBLITERATUS initializes CUDA, maintain the
|
||||
lease for the entire process, and release it after CUDA allocations terminate.
|
||||
|
||||
Schedulers that drain another inference service commonly distinguish `acquire`
|
||||
from `ready`. OBLITERATUS does not yet emit a portable post-model-load readiness
|
||||
signal. Such deployments must not mark a lease ready immediately after process
|
||||
start; add and test an application readiness hook first.
|
||||
|
||||
The requested VRAM must include weights, activation collection, verification,
|
||||
checkpoint snapshots, CUDA context, and a safety margin. Multi-GPU sharding is
|
||||
a capacity feature, not a general throughput multiplier.
|
||||
|
||||
## Filesystem layout
|
||||
|
||||
A conventional layout is:
|
||||
|
||||
```text
|
||||
/srv/obliteratus/
|
||||
├── releases/<commit>/ # administrator-owned source and .venv
|
||||
├── current -> releases/... # atomic promotion target
|
||||
├── users/<login>/ # mode 0700 workspaces and per-user caches
|
||||
├── catalog/ # optional administrator-curated read-only models
|
||||
└── staging/ # release verification before promotion
|
||||
```
|
||||
|
||||
Large model checkpoints and rewritten outputs require a dedicated filesystem.
|
||||
Set capacity and inode alerts, per-user quotas, and a retention policy before
|
||||
allowing downloads. A safe default is no automatic backup of reproducible model
|
||||
caches and outputs; back up only configs, reports, provenance, and explicitly
|
||||
promoted research artifacts.
|
||||
|
||||
## Release and rollback
|
||||
|
||||
1. Select an immutable signed tag or reviewed commit.
|
||||
2. Verify the source artifact and supply-chain evidence described in
|
||||
[the release process](../RELEASE_PROCESS.md).
|
||||
3. Build the locked environment in a new release directory.
|
||||
4. Run `installer/scripts/verify.sh` on the target CUDA host.
|
||||
5. Run a small, pinned-model smoke job through the real GPU scheduler.
|
||||
6. Atomically repoint `current` only after all checks pass.
|
||||
7. Retain the previous release until the observation window ends.
|
||||
|
||||
Rollback is an atomic symlink reversal followed by termination and restart of
|
||||
new work. Do not change the release beneath a running job.
|
||||
|
||||
## Activation gate
|
||||
|
||||
Production activation requires all of the following:
|
||||
|
||||
- dedicated storage mounted and monitored;
|
||||
- named users/groups and private workspace creation tested;
|
||||
- GPU lease integration with a truthful post-load readiness signal;
|
||||
- pinned release and locked CUDA environment verified;
|
||||
- credential isolation and telemetry policy documented;
|
||||
- smoke test, failure test, cancellation test, and rollback test passed;
|
||||
- operator runbook and host inventory published outside the application repo.
|
||||
Reference in New Issue
Block a user