Commit Graph
8 Commits
Author SHA1 Message Date
Stella Biderman a2bb748f1b Revert "Add data parallel support for PROBE stage"
This reverts commit 1a6e2577bb.
2026-03-13 16:54:31 -04:00
Stella Biderman 1a6e2577bb Add data parallel support for PROBE stage
When --data-parallel is passed and the model fits on a single GPU,
wraps it with nn.DataParallel to split prompt batches across all
available GPUs during activation collection. Batch size scales by
GPU count. Hooks already move activations to CPU so they work
correctly across replicas.
2026-03-13 01:24:31 -04:00
Stella Biderman cbdb772eb9 Add multi-GPU support with --gpus flag
Adds --gpus flag to obliterate, run, and tourney commands for controlling
which GPUs to use (sets CUDA_VISIBLE_DEVICES). Works both locally and with
--remote. Models are automatically split across selected GPUs via
accelerate's device_map="auto". Also adds gpus field to remote YAML config.
2026-03-12 12:35:29 -04:00
Stella Biderman 34032b7821 Add remote SSH execution support for GPU nodes
Adds --remote [user@]host flag to obliterate, run, and tourney commands,
enabling execution on remote GPU nodes via SSH. Also supports a remote:
section in YAML configs. The remote runner handles SSH connectivity checks,
GPU detection, auto-installation of obliteratus, log streaming, and result
syncing back to the local machine via scp.
2026-03-11 15:51:16 -04:00
pliny 69fa63ac43 Add files via upload 2026-03-08 12:07:56 -07:00
pliny 1065809658 Add files via upload 2026-03-07 17:54:38 -08:00
pliny 8312bbfa8d Add files via upload 2026-03-04 13:09:57 -08:00
pliny 0f6114fe87 Add files via upload 2026-03-04 12:38:18 -08:00