mirror of
https://github.com/romovpa/claudini.git
synced 2026-08-21 01:07:13 +02:00
SecAlign-70B support: configs, quantization, multi-GPU (#1)
- **PEFT adapter merging.** `model_loader.py` auto-detects PEFT adapters (e.g. `facebook/Meta-SecAlign-8B`), merges on CPU in bf16, and caches the merged model to disk. No config flags needed. - **Configurable quantization.** `quantization:` field in YAML or `--quantization` on CLI, accepting `nf4`, `fp4`, or `int8`. Replaces the old `load_in_4bit` boolean. - **Multi-GPU sharding.** `device_map:` in configs or `--device-map` on CLI. Config value is now correctly read from YAML presets (was previously ignored). - **CLI overrides.** New `--model`, `--device-map`, `--quantization` flags to override preset values from the command line. - **SecAlign injection presets.** Configs for prompt injection on Meta-SecAlign-8B and 70B (default + Optuna-tuned), using new `AlpacaInjectionSource` — generates 3-role prompts from AlpacaFarm data with trusted/untrusted separation. - **Fixes.** `BenchmarkRunner.summarize()` crash when all runs are skipped. System prompt suppression now works correctly (`""` suppresses model defaults, `None` omits the turn). Co-authored-by: Peter Romov <peter@romov.com> Co-authored-by: Alexander Panfilov <39771221+kotekjedi@users.noreply.github.com>
This commit is contained in:
co-authored by
Alexander Panfilov
parent
48eb0f155c
commit
59106bdf3c
+4
-1
@@ -3,7 +3,7 @@ name = "claudini"
|
||||
version = "0.1.0"
|
||||
description = "Claudini: Automated Research Discovers State-of-the-Art Adversarial Attack Algorithms"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
requires-python = ">=3.14"
|
||||
dependencies = [
|
||||
"accelerate>=1.13.0",
|
||||
"numpy>=2.0",
|
||||
@@ -14,6 +14,9 @@ dependencies = [
|
||||
"transformers>=4.40",
|
||||
"typer>=0.9",
|
||||
"datasets>=2.14",
|
||||
"huggingface_hub>=0.20",
|
||||
"bitsandbytes>=0.46.1",
|
||||
"peft>=0.18.1",
|
||||
]
|
||||
|
||||
[build-system]
|
||||
|
||||
Reference in New Issue
Block a user