OBLITERATUS/docs/executive_research_summary.md

# Complementary Abliteration Blending: Executive Research Summary

**OBLITERATUS Project — August 2026**

> **Evidence status:** This summary preserves contributor-reported preliminary measurements from
> PR #127. The PR did not include raw benchmark outputs, prompt-level refusal evidence, exact model
> revisions, environment capture, or artifact hashes. Maintainers independently verified the blend
> implementation and its CPU contracts, but not the numerical research results below.

---

## The Problem

The contributor framed existing abliteration approaches as trading deeper refusal removal for
greater capability loss. Whether that pattern generalizes or follows from refusal-model geometry
has not been established by the evidence included with PR #127.

| Approach | Refusal Rate | MMLU Delta | Source |
|---|---|---|---|
| Single direction (Arditi et al.) | Low but residual | ~0pp | Baseline |
| OrcaRouter (1-dir, k=1) | Low | -0.8pp | Community |
| huihui-ai (1-dir, skip layers) | Low | ~0pp | Community |
| OBLITERATUS V1 (5-dir SVD) | 0.0% | -6.0pp | This work |

The contributor reported that a V1 configuration reached 0% refusal on its sampled prompts with a
-6pp MMLU difference. Those figures are retained as an unverified observation, not proof of a
general tradeoff.

## The Insight

Different direction-finding algorithms damage different regions of weight space.

**SVD (Singular Value Decomposition):** Extracts high-variance directions. The contributor proposes
that some directions encode both refusal and capability; this mechanism was not measured in PR #127.

**LEACE (Linear Erasure of Concept Embeddings):** Provides closed-form linear concept erasure while
minimizing distortion. The reported capability retention and generation-pathway residue remain
contributor observations requiring artifact-backed reproduction.

The working hypothesis is that these methods make complementary errors. The PR does not measure
their error correlation or task-vector geometry.

## The Method

Run both surgeries independently on the same base model, then interpolate in weight space:

```
blended_weight = α × LEACE_weight + (1 - α) × SVD_weight
```

The contributor searched α over {0.30, 0.50, 0.55, 0.60, 0.65, 0.70} using 15-subject MMLU and a
10-prompt usability check, then selected α = 0.60 for Qwen3.8-27B. The small, incomplete search
does not establish a unique optimum.

**Proposed explanation:** interpolation may dilute method-specific damage. This must be tested
against same-method and unrelated-checkpoint blend controls before it is treated as causal.

## Contributor-Reported Results (Not Independently Reproduced)

### Headline

| Model | MMLU (lm-eval, 0-shot) | Refusal Rate | Usable Output |
|---|---|---|---|
| Stock Qwen3.8-27B | 85.3% (n=570) | ~100% | — |
| V1 (aggressive/SVD) | 81.4% (n=285) | 0.0% | 80% |
| **V2 (60/40 blend)** | **86.3% (n=570)** | **0.0%** | **100%** |

The contributor reported +1.1pp MMLU above stock while maintaining complete refusal removal. The
repository does not claim priority or a confirmed capability improvement without reproducible raw
evidence and appropriate statistical comparison.

**Important caveat:** the contributor reports MMLU with `--limit 10` (570 questions, 10 per
subject), below the full benchmark. Without raw outputs and a paired statistical analysis, the
+1.1pp difference is descriptive only and may reflect sampling variation.

### Per-Subject Analysis

Gains span both safety-adjacent and neutral reasoning topics (5 questions per subject — preliminary):

| Subject | Stock | V2 | Delta | Topic Type |
|---|---|---|---|---|
| College Mathematics | 40% | 80% | +40pp | Neutral |
| Formal Logic | 40% | 60% | +20pp | Neutral |
| Jurisprudence | 60% | 80% | +20pp | Safety-adjacent |
| Business Ethics | 80% | 100% | +20pp | Safety-adjacent |
| Professional Law | 80% | 100% | +20pp | Safety-adjacent |
| High School Chemistry | 100% | 80% | -20pp | Neutral (regression) |
| College Computer Science | 80% | 60% | -20pp | Neutral (regression) |

The reported neutral-topic differences motivate a reduced-hedging control, but the per-subject
samples are too small to distinguish a capability effect from sampling variation.

### Practical Capability

| Test Category | V2 | Stock | N |
|---|---|---|---|
| Advanced real-world tasks | 7/8 | 7/8 | 8 |
| Basic real-world tasks | 6/8 | 5/8 | 8 |
| Tool calling (JSON, ReAct) | ✓ | ✓ | — |
| Code generation & refactoring | ✓ | ✓ | — |
| Security code review | ✓ | ✓ | — |
| Structured output (JSON schema) | ✓ | ✓ | — |
| System design | ✓ | ✓ | — |

The contributor reported comparable outcomes on this small practical-task set. The underlying
tasks and outputs were not included for independent review.

## What We Don't Know Yet

### Unanswered Questions

1. **Does the +1.1pp hold at full MMLU scale?** Our sample (570q) is above spot-check but below the full benchmark (14,042q). The number could converge to +0pp or +2pp with more data.

2. **WHY does the blend improve over stock?** Three competing hypotheses:
   - **Freed capacity:** Refusal training occupies representational capacity; removing it frees parameters for reasoning. Gains on math/logic support this.
   - **Reduced hedging:** Stock model hedges on questions adjacent to sensitive topics; abliteration removes the hedging. Gains on law/ethics support this.
   - **Blend regularization:** Weight averaging of any two diverse models acts as implicit regularization (analogous to model soups/ensembling). The improvement may not be specific to abliteration.

3. **Is the 60/40 ratio model-specific?** The contributor reported testing only Qwen3.8-27B.
   Useful ratios may vary by architecture, model size, and alignment training method.

4. **Does this generalize beyond SVD + LEACE?** Other direction-finding methods (diff_means, SOM, nuclear/SAE) may offer additional complementary error profiles for three-way or N-way blends.

5. **What happens with SLERP instead of LERP?** Spherical interpolation preserves weight norms better than linear interpolation. This may matter for models with strong norm-dependent behaviors.

### Validation Gaps

- Full MMLU (14k questions): in progress
- Full 842-corpus refusal validation: in progress (52/842 sample showed 0%)
- MMLU-Pro: not yet run
- Thinking mode ON: not tested
- GGUF inference validation: not tested (GGUFs compiled, not inference-checked)
- Cross-architecture replication: not attempted

## Experimental Framework for Future Validation

Four proposed experiments are tracked in [issue #132](https://github.com/elder-plinius/OBLITERATUS/issues/132):

### Experiment 1: Activation Rank Analysis (Tests "freed capacity")
Run diverse prompts through stock and abliterated models. Capture hidden states at each layer. Compute effective rank via SVD. If abliteration frees capacity, the effective dimensionality of activations should increase.

### Experiment 2: Topic Cluster Analysis (Tests "reduced hedging")
Compare per-subject MMLU differences between prespecified sensitive and neutral groups. The test
must define its statistical decision rule before examining results; the small table above is not
such a test.

### Experiment 3: Blend Control (Tests "blend regularization")
Blend two identical SVD surgeries (same method, different random seeds) at 60/40. If this blend also gains MMLU, the improvement comes from weight averaging itself, not from the SVD/LEACE complementarity. This is the critical control experiment.

### Experiment 4: Learning Absorption (Tests "freed capacity" directly)
QLoRA fine-tune both stock and abliterated models on identical small datasets. Compare loss curves. If the abliterated model learns faster (lower loss at same step count), it has more absorptive capacity — direct evidence for freed representational space. Requires GPU infrastructure (A100+, not feasible on MPS).

Their implementation is intentionally deferred until the hypotheses, controls, provenance, CPU
contracts, and conditional GPU/network gates are specified.

## Future Directions

### Near-term (implementation available; research validation pending)

1. **Cross-architecture replication.** Run the identical pipeline on additional model families.
   This is needed to determine whether the technique generalizes and which parts of the recipe
   require model-specific tuning.

2. **Full-scale benchmarking.** Complete MMLU (14k), MMLU-Pro, HumanEval, GSM8K, and
   ARC-Challenge with pinned inputs, raw results, and uncertainty estimates.

3. **N-way blending.** Test three or more surgeries using different direction methods (SVD,
   LEACE, diff_means, SOM) against prespecified pairwise and same-method controls.

4. **Blend ratio as a function of model properties.** Study how selected α values relate to model
   size, architecture, alignment training intensity, and number of refusal directions.

### Medium-term (theoretical, needs investigation)

5. **Post-blend capability recovery.** A provenance-safe dataset and QLoRA pipeline are proposed in
   [issue #133](https://github.com/elder-plinius/OBLITERATUS/issues/133). No corpus or recovery
   trainer is shipped by this change.

6. **SLERP and task-arithmetic blending.** Replace LERP with spherical interpolation (preserves weight norms) or task-arithmetic approaches (TIES-Merging, DARE) that handle parameter conflicts more intelligently. LERP is the simplest possible blend — there is likely headroom from more sophisticated interpolation.

7. **Adaptive per-layer blending.** Instead of a global α, use a different blend ratio per layer based on that layer's refusal vs capability contribution (measurable via activation probing). Layers with more refusal content get more SVD weight; layers with more capability content get more LEACE weight. This is the "precision blend" extension.

8. **Blend as continuous optimization.** Instead of grid-searching α, treat the blend ratio as a differentiable parameter and optimize it directly against a capability+refusal objective using a small validation set. This is feasible on a single GPU and could find non-uniform per-tensor blend ratios.

### Long-term (speculative, high-impact if true)

9. **Capacity-hypothesis validation.** Increased activation rank alone would not demonstrate freed
   representational capacity; the proposed work must control for prompt sampling, layer selection,
   model identity, numerical thresholds, and alternative explanations before drawing implications
   about safety training.

10. **Generalized complementary merging.** The principle — "combine models that fail in different ways" — may extend beyond abliteration to any model merging scenario. Fine-tunes optimized for different objectives (code, math, reasoning) could be blended using the same complementary error cancellation principle, with direction-specific merge ratios instead of uniform interpolation.

11. **Abliteration as a diagnostic.** If abliterated models consistently show capability changes on specific subjects, the per-subject delta profile becomes a map of where safety training allocated capacity. This "refusal cost map" could inform alignment researchers about which capabilities are most affected by safety training and guide more efficient alignment methods.

---

## Reproduction

```bash
# Step 1: Aggressive/SVD surgery
obliteratus obliterate $BASE --method aggressive --n-directions 3 \
  --regularization 0.08 --residue-weight 3 --refinement-passes 2 \
  --min-layer-fraction 0.45 --output-dir surgery_svd

# Step 2: LEACE surgery
obliteratus obliterate $BASE --method aggressive --direction-method leace \
  --n-directions 3 --regularization 0.06 --residue-weight 7 \
  --refinement-passes 3 --min-layer-fraction 0.40 --output-dir surgery_leace

# Step 3: Blend
obliteratus blend --model-a surgery_svd --model-b surgery_leace \
  --alpha 0.6 --config-source a --output blended

# Step 4: Validate
lm_eval --model hf --model_args pretrained=blended --tasks mmlu --device auto
```

All code is open source: [github.com/elder-plinius/OBLITERATUS](https://github.com/elder-plinius/OBLITERATUS)

Primary background: [Arditi et al.](https://arxiv.org/abs/2406.11717),
[LEACE](https://arxiv.org/abs/2306.03819),
[Model Soups](https://proceedings.mlr.press/v162/wortsman22a.html), and
[TIES-Merging](https://arxiv.org/abs/2306.01708).

---

*OBLITERATUS Contributors, August 2026*