mirror of
https://github.com/elder-plinius/OBLITERATUS.git
synced 2026-08-30 14:40:38 +02:00
feat: add tune-only Qwen3.8 E02 candidate (#185)
This commit is contained in:
@@ -2,15 +2,18 @@
|
||||
|
||||
## Decision summary
|
||||
|
||||
The current Qwen3.8-27B result is a useful capability-preserving control, not a
|
||||
successful refusal-removal result:
|
||||
The promotion-grade E01 run is a useful causal control, not a successful
|
||||
refusal-removal result:
|
||||
|
||||
- refusal rate: 90.9% (30/33 still refusing);
|
||||
- coherence: 80% (8/10);
|
||||
- perplexity: 3.11;
|
||||
- first-token KL divergence: 0.2405;
|
||||
- 39 middle-depth layers and 312 matrices modified;
|
||||
- 13 layers partially reverted by the current correction pass.
|
||||
- archive run: `run-4787f0f2c2ee4a6b8ca8840684eff99b`;
|
||||
- refusal rate: 92% (184/200 still refusing);
|
||||
- coherence: 70% (7/10);
|
||||
- capability: 83% (5/6 checks);
|
||||
- perplexity: 3.06;
|
||||
- sequence-token KL divergence: 0.0513;
|
||||
- first-token KL divergence: 0.1235;
|
||||
- 62 layers and 62 attention-output matrices modified;
|
||||
- checkpoint: 53,812,173,294 bytes, 35 files, all SHA-256 inventoried.
|
||||
|
||||
Do not increase surgery strength or enable the full advanced toggle set yet.
|
||||
The next engineering release should make the optimizer and evaluation reliable,
|
||||
@@ -211,7 +214,7 @@ method needs an isolated ablation and an interaction test before composition.
|
||||
|---|---|---|---|---|
|
||||
| E00 | Is the pristine evaluator stable? | final-test only, 3 seeds | none | metric variance and baselines recorded |
|
||||
| E01 | Does the Arditi control work? | 400/100 | DIM, one layer/direction sweep | lower refusal with <=1.25x PPL |
|
||||
| E02 | How much data is needed? | 33/64/128/256/400 | fixed DIM/SVD setup | direction and outcome stability plateau |
|
||||
| E02 | Does a low-rank writer intervention improve E01? | 500/142; final test prohibited | SVD-4 + RDO, middle-60% residual writers, KL rollback | tune refusal <30% and coherence >=80% |
|
||||
| E03 | Which architecture component carries refusal? | 400/100 | DeltaNet vs full-attn vs MLP groups | best causal effect per KL unit |
|
||||
| E04 | Does SVD rank help? | 400/100 | k=1..7, joint held-out selection | nondominated gain over E01 |
|
||||
| E05 | Does RDO improve targeting? | 400/100 | one and multiple RDO directions | held-out gain over E01/E04 |
|
||||
@@ -253,12 +256,11 @@ from 60% refusal in S2 to 100% in several other strata.
|
||||
|
||||
## Recommended next operational run
|
||||
|
||||
Do not treat another 54 GB permanent checkpoint as the next experiment. After
|
||||
P0 and hook-mode P1 exist, run E00--E03 first. If an immediate pre-fix diagnostic
|
||||
is unavoidable, use the last stable settings but increase direction-training
|
||||
volume to at least 256 pairs, keep all destructive optional techniques off, and
|
||||
do not promote or publish the output because the current KL and spectral gates
|
||||
are not trustworthy enough for release qualification.
|
||||
Run the pre-registered E02 candidate on the 142-pair optimizer-tune partition.
|
||||
E02 must not read the 200-pair final partition. Retain its complete archive even
|
||||
if it fails. If it misses either target, use only tune evidence to define the
|
||||
next isolated ablation. If it passes both targets, reload that saved checkpoint
|
||||
and run the final partition exactly once for release qualification.
|
||||
|
||||
## Evidence base
|
||||
|
||||
|
||||
@@ -42,8 +42,20 @@
|
||||
},
|
||||
{
|
||||
"id": "E02",
|
||||
"purpose": "direction training-volume ablation",
|
||||
"depends_on": "E01"
|
||||
"purpose": "tune-only multi-direction residual-writer candidate",
|
||||
"depends_on": "E01",
|
||||
"method": "qwen38_e02",
|
||||
"evaluation_split": "optimizer_tune",
|
||||
"direction_method": "svd",
|
||||
"directions": 4,
|
||||
"layer_selection": "middle60",
|
||||
"projection_target": "output",
|
||||
"regularization": 0.0,
|
||||
"norm_preserve": true,
|
||||
"refinement_passes": 1,
|
||||
"rdo_refinement": true,
|
||||
"winsorize_activations": true,
|
||||
"kl_optimization": true
|
||||
},
|
||||
{
|
||||
"id": "E03",
|
||||
|
||||
Reference in New Issue
Block a user