mirror of
https://github.com/elder-plinius/OBLITERATUS.git
synced 2026-08-30 06:30:37 +02:00
233 lines
10 KiB
Markdown
233 lines
10 KiB
Markdown
# Complementary Abliteration Blending
|
|
|
|
**Date:** 2026-08-20
|
|
**Status:** Contributor-reported preliminary results; implementation independently tested
|
|
**Authors:** OBLITERATUS contributors and maintainers
|
|
|
|
---
|
|
|
|
## Abstract
|
|
|
|
This document describes **complementary abliteration blending**, which combines two compatible
|
|
abliterated checkpoints via weight-space interpolation. PR #127 did not include the raw benchmark
|
|
outputs, prompt-level refusal evidence, model revisions, environment capture, or artifact hashes
|
|
needed to independently reproduce its numerical results. The tables below therefore preserve the
|
|
contributor's preliminary report as context; they are not independently verified project claims.
|
|
|
|
---
|
|
|
|
## 1. Motivation
|
|
|
|
The contributor report frames abliteration as a tradeoff between refusal removal and measured
|
|
capability. Single-direction work such as [Arditi et al.][arditi] motivates the approach, while the
|
|
reported OBLITERATUS comparison observed lower MMLU for a more aggressive surgery. Raw evidence
|
|
for the specific refusal and MMLU figures was not included in PR #127.
|
|
|
|
We hypothesized that different direction-finding methods damage different parts of the model's
|
|
capability geometry, and that blending their outputs could cancel these damages.
|
|
|
|
---
|
|
|
|
## 2. Method
|
|
|
|
### 2.1 Surgery A: Aggressive/SVD
|
|
|
|
Standard OBLITERATUS aggressive pipeline with SVD-based direction extraction:
|
|
|
|
```
|
|
obliteratus obliterate $BASE --method aggressive --n-directions 3 \
|
|
--regularization 0.08 --residue-weight 3 --refinement-passes 2 \
|
|
--min-layer-fraction 0.45
|
|
```
|
|
|
|
**Contributor-reported properties:** deep refusal removal with measurable capability damage on a
|
|
small spot check. The proposed explanation—SVD directions overlapping capability subspaces—is a
|
|
hypothesis that requires activation and controlled model-comparison evidence.
|
|
|
|
### 2.2 Surgery B: LEACE
|
|
|
|
OBLITERATUS aggressive pipeline with LEACE (Linear Erasure of Concept Embeddings) direction method:
|
|
|
|
```
|
|
obliteratus obliterate $BASE --method aggressive --direction-method leace \
|
|
--n-directions 3 --regularization 0.06 --residue-weight 7 \
|
|
--refinement-passes 3 --min-layer-fraction 0.40
|
|
```
|
|
|
|
LEACE is a closed-form linear concept-erasure method designed to remove linearly available concept
|
|
information while minimizing distortion ([Belrose et al.][leace]). The contributor reported
|
|
capability retention and weaker output quality for this surgery; the asserted mechanism in the
|
|
generation pathway was not evidenced in this PR.
|
|
|
|
### 2.3 Weight-Space LERP Blend
|
|
|
|
Simple linear interpolation in weight space:
|
|
|
|
```python
|
|
for key in weight_keys:
|
|
blended[key] = alpha * leace_weights[key] + (1 - alpha) * aggressive_weights[key]
|
|
```
|
|
|
|
The blend ratio `alpha = 0.60` was found by binary search over {0.30, 0.50, 0.55, 0.60, 0.65, 0.70}.
|
|
|
|
---
|
|
|
|
## 3. Contributor-Reported Results (Not Independently Reproduced)
|
|
|
|
No machine-readable benchmark or refusal-evaluation artifacts accompany these tables. Counts,
|
|
uncertainty, prompt selection, exact model revisions, and evaluator configuration must be supplied
|
|
before the results can support a release or research conclusion.
|
|
|
|
### 3.1 Blend Ratio Search (15-subject MMLU, 150 questions)
|
|
|
|
| Blend (% LEACE) | Refuse | Usable | MMLU | vs Stock |
|
|
|------------------|--------|--------|-------|----------|
|
|
| 0% (pure SVD) | 0% | 100% | 84.0% | -2.0pp |
|
|
| 30% | 0% | 90% | 83.3% | -2.7pp |
|
|
| 50% | 0% | 100% | 84.0% | -2.0pp |
|
|
| 55% | 0% | 80% | 84.7% | -1.3pp |
|
|
| **60%** | **0%** |**100%**|**86.0%**|**+0.0pp**|
|
|
| 65% | 0% | — | — | — |
|
|
| 70% | 0% | 80% | 86.7% | +0.7pp |
|
|
| 100% (pure LEACE)| 0% | 50% | 86.7% | +0.7pp |
|
|
|
|
The contributor selected the 60% blend from this small search. The evidence is insufficient to
|
|
establish a unique optimum or distinguish the apparent differences from sampling noise.
|
|
|
|
### 3.2 Larger Contributor Spot Check (57-subject MMLU, 570 questions)
|
|
|
|
| Model | MMLU | Stderr | vs Stock |
|
|
|----------------------|---------|--------|----------|
|
|
| Stock Qwen3.8-27B | 85.26% | ±0.014 | — |
|
|
| OBLITERATUS V1 (s51) | 81.40% | — | -6.0pp |
|
|
| **OBLITERATUS V2 (s78)** | **86.32%** | **±0.014** | **+1.1pp** |
|
|
|
|
### 3.3 Per-Subject Gains (5 questions/subject)
|
|
|
|
Capability gains span both safety-adjacent and neutral reasoning topics:
|
|
|
|
| Subject | Stock | V2 | Delta | Type |
|
|
|----------------------|-------|-------|--------|-----------|
|
|
| College Mathematics | 40% | 80% | +40pp | Neutral |
|
|
| Formal Logic | 40% | 60% | +20pp | Neutral |
|
|
| Jurisprudence | 60% | 80% | +20pp | Sensitive |
|
|
| Business Ethics | 80% | 100% | +20pp | Sensitive |
|
|
| Professional Law | 80% | 100% | +20pp | Sensitive |
|
|
|
|
The reported neutral-topic gains motivate a controlled follow-up; five questions per subject are
|
|
not sufficient to establish capability improvement or rule out sampling variation.
|
|
|
|
### 3.4 Real-World Practical Tasks
|
|
|
|
| Test Suite | Stock | V2 | Tasks |
|
|
|---------------------|-------|-----|------------------------------------------|
|
|
| Basic (8 tasks) | 5/8 | 6/8 | Code, SQL, tool calling, JSON, math |
|
|
| Advanced (8 tasks) | 7/8 | 7/8 | ReAct agents, async refactor, K8s debug, |
|
|
| | | | security review, system design |
|
|
|
|
The contributor reported comparable outcomes on this small practical-task set. The tasks and raw
|
|
outputs were not included, so maintainers have not independently verified that comparison.
|
|
|
|
---
|
|
|
|
## 4. Why It Works
|
|
|
|
### 4.1 Complementary Error Cancellation
|
|
|
|
One hypothesis is that SVD and LEACE make different errors in weight space:
|
|
|
|
- **SVD** greedily captures maximum variance directions. Some captured variance encodes
|
|
capability, not just refusal. This damages specific weight regions.
|
|
- **LEACE** minimizes a linear erasure objective. Whether it leaves specific residue in attention
|
|
heads or output projections must be measured rather than inferred from output behavior.
|
|
|
|
Weight-space interpolation may average complementary errors:
|
|
- Where SVD damaged capability, LEACE's intact weights dilute the damage
|
|
- Where LEACE left refusal residue, SVD's clean weights dilute the residue
|
|
|
|
### 4.2 Theoretical Connection to Model Merging
|
|
|
|
This technique is related to model-merging work such as [Model Soups][model-soups] and
|
|
[TIES-Merging][ties], but is applied within the abliteration domain. PR #127 does not measure
|
|
task-vector orthogonality or error anti-correlation, so those remain testable explanations rather
|
|
than established properties.
|
|
|
|
### 4.3 Capacity Hypothesis
|
|
|
|
The contributor-reported MMLU difference raises several competing hypotheses. A separate,
|
|
provenance-gated experiment framework is tracked in [issue #132][capacity-issue]:
|
|
|
|
1. **Activation Rank Analysis** — Does effective dimensionality increase after abliteration?
|
|
2. **Topic Cluster Analysis** — Do gains cluster on sensitive topics (hedging) or spread broadly (capacity)?
|
|
3. **Blend Control** — Does blending two identical SVD surgeries also gain MMLU? (Tests regularization hypothesis)
|
|
4. **Learning Absorption** — Does the abliterated model learn new information faster? (Tests freed capacity directly)
|
|
|
|
The small per-subject report does not distinguish these hypotheses.
|
|
|
|
---
|
|
|
|
## 5. Reproducibility
|
|
|
|
```bash
|
|
# Step 1: Aggressive surgery
|
|
obliteratus obliterate $BASE --method aggressive --n-directions 3 \
|
|
--regularization 0.08 --residue-weight 3 --refinement-passes 2 \
|
|
--min-layer-fraction 0.45 --output-dir surgery_a
|
|
|
|
# Step 2: LEACE surgery
|
|
obliteratus obliterate $BASE --method aggressive --direction-method leace \
|
|
--n-directions 3 --regularization 0.06 --residue-weight 7 \
|
|
--refinement-passes 3 --min-layer-fraction 0.40 --output-dir surgery_b
|
|
|
|
# Step 3: Blend
|
|
obliteratus blend --model-a surgery_a --model-b surgery_b --alpha 0.6 \
|
|
--config-source a --output blended_model
|
|
|
|
# Step 4: Validate
|
|
lm_eval --model hf --model_args pretrained=blended_model --tasks mmlu
|
|
```
|
|
|
|
The command requires matching `source_model` values in both checkpoints'
|
|
`abliteration_metadata.json`, identical tensor keys, compatible shapes/dtypes, floating-point
|
|
weights, and sharded safetensors indexes. For legacy checkpoints whose common lineage was verified
|
|
out of band, `--allow-unverified-lineage` is an explicit escape hatch. Output is staged and
|
|
validated before atomically replacing any existing destination.
|
|
|
|
---
|
|
|
|
## 6. Limitations
|
|
|
|
- Contributor measurements cover only Qwen3.8-27B; independent validation is pending
|
|
- MMLU is a multiple-choice benchmark; gains may not transfer to all downstream tasks
|
|
- The 60/40 blend ratio may be model-specific
|
|
- `repetition_penalty=1.15` is still required for clean generation
|
|
- System prompts still reintroduce refusals
|
|
- Full 842-corpus validation in progress at time of writing
|
|
- The numerical results lack committed raw evidence and independent reproduction
|
|
- Single-file and quantized/integer checkpoints are not supported by the current blender
|
|
- Atomic promotion temporarily requires space for the complete staged output and any prior output
|
|
|
|
---
|
|
|
|
## 7. Future Work
|
|
|
|
- **Cross-architecture validation** on Llama, Gemma, Mistral
|
|
- **SLERP blending** instead of LERP (spherical interpolation may better preserve weight norms)
|
|
- **Three-way blends** with additional direction methods (diff_means, SOM)
|
|
- **Post-blend recovery**, tracked separately in [issue #133][recovery-issue]
|
|
- **Formal capacity-hypothesis validation**, tracked in [issue #132][capacity-issue]
|
|
|
|
## References
|
|
|
|
- [Arditi et al., *Refusal in Language Models Is Mediated by a Single Direction*][arditi]
|
|
- [Belrose et al., *LEACE: Perfect Linear Concept Erasure in Closed Form*][leace]
|
|
- [Wortsman et al., *Model Soups*][model-soups]
|
|
- [Yadav et al., *TIES-Merging*][ties]
|
|
|
|
[arditi]: https://arxiv.org/abs/2406.11717
|
|
[leace]: https://arxiv.org/abs/2306.03819
|
|
[model-soups]: https://proceedings.mlr.press/v162/wortsman22a.html
|
|
[ties]: https://arxiv.org/abs/2306.01708
|
|
[capacity-issue]: https://github.com/elder-plinius/OBLITERATUS/issues/132
|
|
[recovery-issue]: https://github.com/elder-plinius/OBLITERATUS/issues/133
|