Sample 0 of ClearHarm appears in training/eval pretraining data we
want to keep clean of. Replace it with sample 51 to keep N=40.
Assisted-by: Claude <noreply@anthropic.com>
* safeguard_valid: drop 10 samples that trip gpt-oss attention bug
Exclude {2, 4, 5, 12, 14, 15, 28, 33, 44, 47} which crash with a
170-vs-169 size mismatch in eager_attention_forward under the
clearharm template. Matches hmcGCG's safeguard_clearharm_gpt-oss_3e17
sample set.
Assisted-by: Claude <noreply@anthropic.com>
* Add claudini.asr: compute ASR from benchmark results
ASR (Attack Success Rate) is the fraction of runs whose greedy
completion from the best suffix exactly matches every target
token (i.e. match_rate == 1.0). This is the canonical success
metric for the safeguard track, distinct from loss-based ranking.
uv run -m claudini.asr results/ --preset safeguard_valid
Assisted-by: Claude <noreply@anthropic.com>
---------
Co-authored-by: Alexander Panfilov <apanfilov@g003.internal.cluster.is.localnet>
Add .claude/skills/claudini/SKILL.md to drive the autoresearch loop
via /claudini slash command. Update CLAUDE.md with skill docs. Replace
PROMPT.txt with the skill-based workflow. Rewrite README to feature
the autoresearch loop prominently. Add easy_1e16 and easy_1e17 preset
configs and update safeguard configs.
Assisted-by: Claude <noreply@anthropic.com>
kotekjedi