mirror of
https://github.com/romovpa/claudini.git
synced 2026-07-11 22:46:33 +02:00
Peter Romov
59106bdf3c
- **PEFT adapter merging.** `model_loader.py` auto-detects PEFT adapters (e.g. `facebook/Meta-SecAlign-8B`), merges on CPU in bf16, and caches the merged model to disk. No config flags needed. - **Configurable quantization.** `quantization:` field in YAML or `--quantization` on CLI, accepting `nf4`, `fp4`, or `int8`. Replaces the old `load_in_4bit` boolean. - **Multi-GPU sharding.** `device_map:` in configs or `--device-map` on CLI. Config value is now correctly read from YAML presets (was previously ignored). - **CLI overrides.** New `--model`, `--device-map`, `--quantization` flags to override preset values from the command line. - **SecAlign injection presets.** Configs for prompt injection on Meta-SecAlign-8B and 70B (default + Optuna-tuned), using new `AlpacaInjectionSource` — generates 3-role prompts from AlpacaFarm data with trusted/untrusted separation. - **Fixes.** `BenchmarkRunner.summarize()` crash when all runs are skipped. System prompt suppression now works correctly (`""` suppresses model defaults, `None` omits the turn). Co-authored-by: Peter Romov <peter@romov.com> Co-authored-by: Alexander Panfilov <39771221+kotekjedi@users.noreply.github.com>
20 lines
468 B
YAML
20 lines
468 B
YAML
# Easy random targets track — 1e16 FLOP budget.
|
|
# Lower budget variant for quick evaluation. No model specified — results exist for gpt2.
|
|
optim_length: 20
|
|
max_flops: 1.0e+16
|
|
dtype: bfloat16
|
|
system_prompt: "" # suppress model-default system prompt (e.g. Qwen's "You are Qwen…")
|
|
samples: [0, 1, 2, 3, 4]
|
|
seeds: [0]
|
|
final_input: tokens
|
|
|
|
input_spec:
|
|
source:
|
|
type: random
|
|
query_len: 0
|
|
target_len: 10
|
|
layout:
|
|
type: suffix
|
|
init:
|
|
type: random
|