feat: add tune-only Qwen3.8 E02 candidate (#185)

This commit is contained in:
Joseph Magly
2026-08-29 03:54:53 -04:00
parent 312e63dd00
commit 941869d909
7 changed files with 170 additions and 50 deletions
+17 -15
View File
@@ -2,15 +2,18 @@
## Decision summary
The current Qwen3.8-27B result is a useful capability-preserving control, not a
successful refusal-removal result:
The promotion-grade E01 run is a useful causal control, not a successful
refusal-removal result:
- refusal rate: 90.9% (30/33 still refusing);
- coherence: 80% (8/10);
- perplexity: 3.11;
- first-token KL divergence: 0.2405;
- 39 middle-depth layers and 312 matrices modified;
- 13 layers partially reverted by the current correction pass.
- archive run: `run-4787f0f2c2ee4a6b8ca8840684eff99b`;
- refusal rate: 92% (184/200 still refusing);
- coherence: 70% (7/10);
- capability: 83% (5/6 checks);
- perplexity: 3.06;
- sequence-token KL divergence: 0.0513;
- first-token KL divergence: 0.1235;
- 62 layers and 62 attention-output matrices modified;
- checkpoint: 53,812,173,294 bytes, 35 files, all SHA-256 inventoried.
Do not increase surgery strength or enable the full advanced toggle set yet.
The next engineering release should make the optimizer and evaluation reliable,
@@ -211,7 +214,7 @@ method needs an isolated ablation and an interaction test before composition.
|---|---|---|---|---|
| E00 | Is the pristine evaluator stable? | final-test only, 3 seeds | none | metric variance and baselines recorded |
| E01 | Does the Arditi control work? | 400/100 | DIM, one layer/direction sweep | lower refusal with <=1.25x PPL |
| E02 | How much data is needed? | 33/64/128/256/400 | fixed DIM/SVD setup | direction and outcome stability plateau |
| E02 | Does a low-rank writer intervention improve E01? | 500/142; final test prohibited | SVD-4 + RDO, middle-60% residual writers, KL rollback | tune refusal <30% and coherence >=80% |
| E03 | Which architecture component carries refusal? | 400/100 | DeltaNet vs full-attn vs MLP groups | best causal effect per KL unit |
| E04 | Does SVD rank help? | 400/100 | k=1..7, joint held-out selection | nondominated gain over E01 |
| E05 | Does RDO improve targeting? | 400/100 | one and multiple RDO directions | held-out gain over E01/E04 |
@@ -253,12 +256,11 @@ from 60% refusal in S2 to 100% in several other strata.
## Recommended next operational run
Do not treat another 54 GB permanent checkpoint as the next experiment. After
P0 and hook-mode P1 exist, run E00--E03 first. If an immediate pre-fix diagnostic
is unavoidable, use the last stable settings but increase direction-training
volume to at least 256 pairs, keep all destructive optional techniques off, and
do not promote or publish the output because the current KL and spectral gates
are not trustworthy enough for release qualification.
Run the pre-registered E02 candidate on the 142-pair optimizer-tune partition.
E02 must not read the 200-pair final partition. Retain its complete archive even
if it fails. If it misses either target, use only tune evidence to define the
next isolated ablation. If it passes both targets, reload that saved checkpoint
and run the final partition exactly once for release qualification.
## Evidence base
+14 -2
View File
@@ -42,8 +42,20 @@
},
{
"id": "E02",
"purpose": "direction training-volume ablation",
"depends_on": "E01"
"purpose": "tune-only multi-direction residual-writer candidate",
"depends_on": "E01",
"method": "qwen38_e02",
"evaluation_split": "optimizer_tune",
"direction_method": "svd",
"directions": 4,
"layer_selection": "middle60",
"projection_target": "output",
"regularization": 0.0,
"norm_preserve": true,
"refinement_passes": 1,
"rdo_refinement": true,
"winsorize_activations": true,
"kl_optimization": true
},
{
"id": "E03",