mirror of
https://github.com/romovpa/claudini.git
synced 2026-04-22 18:15:58 +02:00
Peter Romov
59106bdf3c
- **PEFT adapter merging.** `model_loader.py` auto-detects PEFT adapters (e.g. `facebook/Meta-SecAlign-8B`), merges on CPU in bf16, and caches the merged model to disk. No config flags needed. - **Configurable quantization.** `quantization:` field in YAML or `--quantization` on CLI, accepting `nf4`, `fp4`, or `int8`. Replaces the old `load_in_4bit` boolean. - **Multi-GPU sharding.** `device_map:` in configs or `--device-map` on CLI. Config value is now correctly read from YAML presets (was previously ignored). - **CLI overrides.** New `--model`, `--device-map`, `--quantization` flags to override preset values from the command line. - **SecAlign injection presets.** Configs for prompt injection on Meta-SecAlign-8B and 70B (default + Optuna-tuned), using new `AlpacaInjectionSource` — generates 3-role prompts from AlpacaFarm data with trusted/untrusted separation. - **Fixes.** `BenchmarkRunner.summarize()` crash when all runs are skipped. System prompt suppression now works correctly (`""` suppresses model defaults, `None` omits the turn). Co-authored-by: Peter Romov <peter@romov.com> Co-authored-by: Alexander Panfilov <39771221+kotekjedi@users.noreply.github.com>
28 lines
570 B
YAML
28 lines
570 B
YAML
# Random targets track validation set — default hyperparams.
|
|
optim_length: 15
|
|
max_flops: 1.0e+17
|
|
dtype: bfloat16
|
|
system_prompt: "" # suppress model-default system prompt (e.g. Qwen's "You are Qwen…")
|
|
samples: [5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24]
|
|
seeds: [0]
|
|
final_input: tokens
|
|
use_prefix_cache: true
|
|
model: Qwen/Qwen2.5-7B-Instruct
|
|
|
|
input_spec:
|
|
source:
|
|
type: random
|
|
query_len: 0
|
|
target_len: 10
|
|
layout:
|
|
type: suffix
|
|
init:
|
|
type: random
|
|
|
|
methods:
|
|
- gcg
|
|
- i_gcg
|
|
- tao
|
|
- claude_v63
|
|
- claude_v82
|