mirror of
https://github.com/elder-plinius/OBLITERATUS.git
synced 2026-07-13 07:36:33 +02:00
Add ASPA framework, AutoObliterator, Watchtower, expanded eval corpus
New core modules: - auto_obliterate.py: Automated multi-iteration obliteration pipeline - watchtower.py: HF Hub model discovery and tracking - ui_watchtower.py: Gradio tabs for Watchtower (ready for app.py wiring) - hard_negative.py: Residue mining from refusal audits - model_profile.py: Parameter profiling from safetensors/config - bestiary_sync.py: Sync models from PlinyOS BESTIARY registry - models_client.py: Lightweight HF model list client Framework enhancements: - abliterate.py: ASPA source-tethering, step gradient blending, hard-negative residue support - cli.py: self-improve command, model profiling, hard-negative flags - prompts.py: Expanded 842-prompt refusal eval corpus across 10 categories - __init__.py: New exports (Watchtower, AutoObliterator) Reference implementations (14 scripts): - ASPA sweep, gradient search, coherence eval, MMLU benchmarks - Pareto controller, refusal sniper, stock comparisons Documentation: - README: Research framing, responsible use section, comprehensive disclaimer - docs/beyond_sota_roadmap.md, docs/recursive_self_improvement.md Tests: 4 new test files (354 lines) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
faber
@@ -0,0 +1,211 @@
|
||||
# Beyond-SOTA Obliteratus Roadmap
|
||||
|
||||
Date: 2026-05-13
|
||||
|
||||
## Current Position
|
||||
|
||||
The current leader is:
|
||||
|
||||
```text
|
||||
outputs/qwen3.6-27b-aspa-n2-reg05-srcgamma090
|
||||
```
|
||||
|
||||
Confirmed metrics:
|
||||
|
||||
| Axis | Result |
|
||||
|---|---:|
|
||||
| Full n120 ship score | 88.392 |
|
||||
| Full refusal rate | 0.025 |
|
||||
| Full usable rate | 0.9667 |
|
||||
| Benign/style pass | 1.0 / 1.0 |
|
||||
| Capability score | 96.75 |
|
||||
| Coding/creative pass | 1.0 / 1.0 |
|
||||
| Boundary score | 100.0 |
|
||||
| Adversarial boundary score | 99.165 |
|
||||
| Full community scrutiny score | 99.5 |
|
||||
| Full community code/knowledge/math/truth | 1.0 / 1.0 / 1.0 / 1.0 |
|
||||
| Full community allowed-topic pass | 1.0 |
|
||||
| Full community restricted refusal/usable | 0.0333 / 0.9667 |
|
||||
| First-token mean KL vs source | 0.549806 |
|
||||
| First-token top-1 agreement vs source | 0.818 |
|
||||
|
||||
The central discovery is Source-Tethered Edit Scaling:
|
||||
|
||||
```text
|
||||
candidate_gamma = source + gamma * (edited_candidate - source)
|
||||
```
|
||||
|
||||
For Qwen3.6-27B, `gamma=0.90` is a real Pareto improvement over raw ASPA:
|
||||
lower refusal, higher usable rate, restored coding pass, and strong boundary
|
||||
behavior.
|
||||
|
||||
The community scrutiny gate adds a broader release-readiness check on top of
|
||||
the original ship/capability probes. The leader passes the full MLX-backed
|
||||
gate with score `99.5`, perfect executable-code, knowledge, math,
|
||||
truthfulness, instruction, creative, and allowed stigmatized-topic results, and
|
||||
restricted aggregate refusal/usable of `0.0333` / `0.9667`.
|
||||
|
||||
The first fine-bracket optimizer pass found:
|
||||
|
||||
| Gamma | Status |
|
||||
|---:|---|
|
||||
| 0.875 | Preservation challenger: capability `96.80`, mean KL `0.525216`, full score `87.250` |
|
||||
| 0.900 | Release leader: capability `96.75`, mean KL `0.549806`, full score `88.392` |
|
||||
| 0.925 | Dominated in this round: capability `96.45`, mean KL `0.589938`, n30 score `88.167` |
|
||||
| 0.875 + late0.90 | Capability-only gain: capability `96.81`, n30 score `86.333` |
|
||||
| 0.875 + late MLP-down0.90 | Component neutral: capability `96.80`, n30 score `88.167` |
|
||||
| 0.875 + late full-attn-o0.90 | Component neutral: capability `96.80`, n30 score `88.167` |
|
||||
| 0.875 + late linear-attn-out0.90 | Component neutral: capability `96.80`, n30 score `88.167` |
|
||||
|
||||
This implies the global scalar is too blunt, but the first broad layer-weighted
|
||||
attempt was also too blunt. `0.875` globally plus `0.90` on layers `48:64`
|
||||
improved capability but worsened n30 refusal. Narrowing that bump to late MLP
|
||||
`down_proj`, late full-attention `self_attn.o_proj`, or late linear-attention
|
||||
`linear_attn.out_proj` preserved capability but did not recover the `0.90`
|
||||
refusal win. The next beyond-SOTA step is a learned component-weighted STES
|
||||
schedule:
|
||||
|
||||
```text
|
||||
candidate = source + gamma(layer, component) * (edited_candidate - source)
|
||||
```
|
||||
|
||||
where preservation-sensitive components use a smaller gamma and
|
||||
refusal-critical components use a larger gamma.
|
||||
|
||||
## What Others Still Have That We Need
|
||||
|
||||
1. **Automated search:** Heretic-style Optuna/TPE loops over edit strength,
|
||||
layer ranges, regularization, and preservation metrics.
|
||||
2. **KL as a first-class objective:** not just a report after the fact, but a
|
||||
constraint or penalty during candidate selection.
|
||||
3. **Quantization-native validation:** edit-then-quantize and quantize-then-edit
|
||||
behavior across MLX/GGUF/JANG-style deployment formats.
|
||||
4. **MoE/router-aware editing:** expert-local edits, router drift checks, and
|
||||
per-expert preservation metrics.
|
||||
5. **Standard public benchmarks:** the local community gate now covers compact
|
||||
HumanEval/MBPP/GSM8K/MMLU-style probes, but we still need full harness
|
||||
exports for public comparability.
|
||||
6. **Judge/calibrated boundary scoring:** transparent heuristics are useful, but
|
||||
a second calibrated local judge should catch both false refusals and unsafe
|
||||
compliance more robustly.
|
||||
|
||||
## Beyond-SOTA Plan
|
||||
|
||||
### Phase 1: Preservation-Aware Search
|
||||
|
||||
Implement an optimizer that treats every candidate as a vector:
|
||||
|
||||
```text
|
||||
score = ship_gate + capability + boundary - KL_penalty - degeneration_penalty
|
||||
```
|
||||
|
||||
Required experiment grid:
|
||||
|
||||
| Family | Values |
|
||||
|---|---|
|
||||
| STES gamma | 0.82, 0.85, 0.875, 0.90, 0.925, 0.95, 0.975 |
|
||||
| component-weighted STES | learned per-component coefficients, residual late 0.90, hybrid sparse masks |
|
||||
| second-pass n-directions | 1, 2, 3 |
|
||||
| regularization | 0.45, 0.50, 0.55, 0.60 |
|
||||
| layer range | final quartile, final third, ASPA current |
|
||||
| component mask | full, MLP out, attention out, hybrid |
|
||||
|
||||
Promotion rule:
|
||||
|
||||
1. n30 ship gate and capability probe for triage.
|
||||
2. KL probe for candidates that pass triage.
|
||||
3. n120 ship gate only for Pareto candidates.
|
||||
4. boundary and adversarial-boundary probe only for release candidates.
|
||||
|
||||
### Phase 2: Edit-Field Tomography
|
||||
|
||||
Measure how each layer/component contributes to:
|
||||
|
||||
- refusal reduction,
|
||||
- benign KL drift,
|
||||
- coding pass/fail cliffs,
|
||||
- boundary over-refusal,
|
||||
- adversarial robustness.
|
||||
|
||||
The output should be an edit-field map, not just a model artifact:
|
||||
|
||||
```text
|
||||
layer -> component -> {refusal_delta, KL_delta, capability_delta, boundary_delta}
|
||||
```
|
||||
|
||||
This is the path beyond generic ablation. The optimizer should learn which
|
||||
subspaces are causal for refusal and which are merely correlated collateral.
|
||||
|
||||
### Phase 3: Quantization Invariance
|
||||
|
||||
Every leader must survive deployment:
|
||||
|
||||
1. BF16 source/candidate metrics.
|
||||
2. MLX or GGUF quantized metrics.
|
||||
3. Quantization drift report:
|
||||
|
||||
```text
|
||||
delta_ship_score
|
||||
delta_capability_score
|
||||
delta_boundary_score
|
||||
delta_first_token_KL
|
||||
```
|
||||
|
||||
Beyond-SOTA target: a candidate is not considered real unless the edit survives
|
||||
the target inference format.
|
||||
|
||||
### Phase 4: MoE and Router Awareness
|
||||
|
||||
For MoE models, add:
|
||||
|
||||
- expert-level direction extraction,
|
||||
- router-logit KL,
|
||||
- per-expert activation KL,
|
||||
- expert-load entropy before/after edit,
|
||||
- expert-local STES rather than one global gamma.
|
||||
|
||||
Beyond-SOTA target:
|
||||
|
||||
```text
|
||||
gamma_expert = optimizer(expert_load, refusal_signal, KL_sensitivity)
|
||||
```
|
||||
|
||||
### Phase 5: Public Scorecard
|
||||
|
||||
Generate a release bundle:
|
||||
|
||||
```text
|
||||
model_card_metrics.json
|
||||
pareto_frontier.json
|
||||
kl_report.json
|
||||
boundary_report.json
|
||||
quantization_report.json
|
||||
benchmark_report.json
|
||||
```
|
||||
|
||||
The model card should show:
|
||||
|
||||
- source model,
|
||||
- edit recipe,
|
||||
- refusal/usable metrics,
|
||||
- over-refusal metrics,
|
||||
- capability benchmarks,
|
||||
- KL preservation,
|
||||
- quantized deployment parity,
|
||||
- known failure cases.
|
||||
|
||||
## Immediate Next Experiments
|
||||
|
||||
1. Promote the MLX-backed eval path as a fallback when torch/MPS is unavailable.
|
||||
2. Run the community gate against source, raw ASPA, `srcgamma875`, and public
|
||||
baseline candidates for side-by-side scrutiny tables.
|
||||
3. Add an optimizer over component regex masks and coefficients, seeded by:
|
||||
late MLP-down, late attention-o, residual/norm, and sparse hybrid masks.
|
||||
4. Add an early objective that can stop dead branches after capability + n30
|
||||
without wasting KL or n120 budget.
|
||||
5. Run KL for raw ASPA, `gamma=0.85`, `gamma=0.95`, and any component candidate
|
||||
that beats `gamma=0.90` on n30 or capability.
|
||||
6. Add a controller objective that ranks full candidates by ship score,
|
||||
capability, boundary score, and low KL.
|
||||
7. Promote only if a candidate beats `gamma=0.90` on at least two axes without losing any
|
||||
hard gate.
|
||||
@@ -0,0 +1,79 @@
|
||||
# Recursive self-improving OBLITERATUS
|
||||
|
||||
OBLITERATUS can now turn refusal audits into hard-negative residue for the next surgery run.
|
||||
|
||||
## Loop
|
||||
|
||||
1. Run an eval/audit that records refused prompt references. The audit should store dataset key, zero- or one-based corpus index, prompt hash, refusal reason, and optionally a short response preview. It does not need to store prompt text.
|
||||
2. Feed the audit back into `obliteratus self-improve`.
|
||||
3. The command profiles the target model without loading full weights. Local safetensors artifacts get exact parameter counts from tensor metadata; Hub/config-only targets use a `text_config`-aware estimate.
|
||||
4. The command writes a canonical residue file and reconstructs prompt text from local dataset registries at run time.
|
||||
5. Residue prompts are appended to the contrastive prompt set with `--residue-weight` copies each. If omitted, `--residue-weight`, `--n-directions`, `--regularization`, `--refinement-passes`, and `--verify-sample-size` are filled from size-aware defaults.
|
||||
6. A `self_improve_plan.json` is written with `model_profile`, `size_aware_defaults`, and `effective_settings` so runs are comparable across parameter scales.
|
||||
7. Run the next artifact through benign/code/math and harmful release gates. Mine its remaining refusals and repeat.
|
||||
|
||||
## Dry-run plan
|
||||
|
||||
```bash
|
||||
obliteratus self-improve outputs/qwen3.6-27b-golden-n3_reg025 \
|
||||
--audit runs/qwen36-refusal-audit/summary.json \
|
||||
--output-dir outputs/qwen3.6-27b-self-improve-r1 \
|
||||
--residue-weight 5 \
|
||||
--residue-max 8 \
|
||||
--method advanced \
|
||||
--direction-method diff_means \
|
||||
--n-directions 3 \
|
||||
--regularization 0.3 \
|
||||
--refinement-passes 1 \
|
||||
--dry-run
|
||||
```
|
||||
|
||||
## Run surgery
|
||||
|
||||
Remove `--dry-run` to launch. The command saves `hard_negative_residue.json` in the output directory with counts, hashes, tags, and indices but not prompt text.
|
||||
|
||||
```bash
|
||||
obliteratus self-improve outputs/qwen3.6-27b-golden-n3_reg025 \
|
||||
--audit runs/qwen36-refusal-audit/summary.json \
|
||||
--output-dir outputs/qwen3.6-27b-self-improve-r1 \
|
||||
--residue-weight 5 \
|
||||
--method advanced \
|
||||
--direction-method diff_means \
|
||||
--n-directions 3 \
|
||||
--regularization 0.3 \
|
||||
--refinement-passes 1
|
||||
```
|
||||
|
||||
Equivalent lower-level usage:
|
||||
|
||||
```bash
|
||||
obliteratus obliterate Qwen/Qwen3.6-27B \
|
||||
--residue-file runs/qwen36-self-improve-dryrun/candidate/mined_residue.json \
|
||||
--residue-weight 5 \
|
||||
--method advanced \
|
||||
--direction-method diff_means \
|
||||
--n-directions 3 \
|
||||
--regularization 0.3 \
|
||||
--refinement-passes 1 \
|
||||
--output-dir outputs/qwen3.6-27b-residue-r1
|
||||
```
|
||||
|
||||
## Current Qwen3.6 residue tags observed
|
||||
|
||||
The Qwen3.6 hard-category audit produced five unique residue prompts after deduping across `n3_reg025` and `n3_reg03`:
|
||||
|
||||
- terrorism + illegal/crime + safety-banner
|
||||
- privacy-intrusion + illegal/crime + safety-banner
|
||||
- sabotage + illegal/crime
|
||||
- CBRN/critical-infra illegal/crime + safety-banner
|
||||
- synthetic-pathogen terrorism + illegal/crime
|
||||
|
||||
These should be upweighted lightly; avoid simply increasing global ablation strength, because nearby regularization sweeps collapsed into short/repetitive `<think>` fragments.
|
||||
|
||||
## Guardrails for future runs
|
||||
|
||||
- Keep `--residue-weight` modest first: omit it to use the size-aware default, or use 3–5 manually.
|
||||
- Preserve a strict benign/code/math gate before expensive harmful eval.
|
||||
- Confirm every plan records `model_profile.total_params_b`; for Qwen3.6-27B local artifacts the expected exact count is about 26.896B from safetensors metadata.
|
||||
- Prefer sequential no-think evaluation on Apple Silicon Qwen3.6.
|
||||
- If residue-weighted runs damage exact/code ability, try source merge-back before stronger ablation.
|
||||
Reference in New Issue
Block a user