Add ASPA framework, AutoObliterator, Watchtower, expanded eval corpus

New core modules:
- auto_obliterate.py: Automated multi-iteration obliteration pipeline
- watchtower.py: HF Hub model discovery and tracking
- ui_watchtower.py: Gradio tabs for Watchtower (ready for app.py wiring)
- hard_negative.py: Residue mining from refusal audits
- model_profile.py: Parameter profiling from safetensors/config
- bestiary_sync.py: Sync models from PlinyOS BESTIARY registry
- models_client.py: Lightweight HF model list client

Framework enhancements:
- abliterate.py: ASPA source-tethering, step gradient blending, hard-negative residue support
- cli.py: self-improve command, model profiling, hard-negative flags
- prompts.py: Expanded 842-prompt refusal eval corpus across 10 categories
- __init__.py: New exports (Watchtower, AutoObliterator)

Reference implementations (14 scripts):
- ASPA sweep, gradient search, coherence eval, MMLU benchmarks
- Pareto controller, refusal sniper, stock comparisons

Documentation:
- README: Research framing, responsible use section, comprehensive disclaimer
- docs/beyond_sota_roadmap.md, docs/recursive_self_improvement.md

Tests: 4 new test files (354 lines)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
faber
2026-06-09 03:54:38 -04:00
co-authored by Claude Opus 4.6
parent d6af36f8d3
commit 04b8ec60cb
32 changed files with 9325 additions and 195 deletions
+79
View File
@@ -0,0 +1,79 @@
# Recursive self-improving OBLITERATUS
OBLITERATUS can now turn refusal audits into hard-negative residue for the next surgery run.
## Loop
1. Run an eval/audit that records refused prompt references. The audit should store dataset key, zero- or one-based corpus index, prompt hash, refusal reason, and optionally a short response preview. It does not need to store prompt text.
2. Feed the audit back into `obliteratus self-improve`.
3. The command profiles the target model without loading full weights. Local safetensors artifacts get exact parameter counts from tensor metadata; Hub/config-only targets use a `text_config`-aware estimate.
4. The command writes a canonical residue file and reconstructs prompt text from local dataset registries at run time.
5. Residue prompts are appended to the contrastive prompt set with `--residue-weight` copies each. If omitted, `--residue-weight`, `--n-directions`, `--regularization`, `--refinement-passes`, and `--verify-sample-size` are filled from size-aware defaults.
6. A `self_improve_plan.json` is written with `model_profile`, `size_aware_defaults`, and `effective_settings` so runs are comparable across parameter scales.
7. Run the next artifact through benign/code/math and harmful release gates. Mine its remaining refusals and repeat.
## Dry-run plan
```bash
obliteratus self-improve outputs/qwen3.6-27b-golden-n3_reg025 \
--audit runs/qwen36-refusal-audit/summary.json \
--output-dir outputs/qwen3.6-27b-self-improve-r1 \
--residue-weight 5 \
--residue-max 8 \
--method advanced \
--direction-method diff_means \
--n-directions 3 \
--regularization 0.3 \
--refinement-passes 1 \
--dry-run
```
## Run surgery
Remove `--dry-run` to launch. The command saves `hard_negative_residue.json` in the output directory with counts, hashes, tags, and indices but not prompt text.
```bash
obliteratus self-improve outputs/qwen3.6-27b-golden-n3_reg025 \
--audit runs/qwen36-refusal-audit/summary.json \
--output-dir outputs/qwen3.6-27b-self-improve-r1 \
--residue-weight 5 \
--method advanced \
--direction-method diff_means \
--n-directions 3 \
--regularization 0.3 \
--refinement-passes 1
```
Equivalent lower-level usage:
```bash
obliteratus obliterate Qwen/Qwen3.6-27B \
--residue-file runs/qwen36-self-improve-dryrun/candidate/mined_residue.json \
--residue-weight 5 \
--method advanced \
--direction-method diff_means \
--n-directions 3 \
--regularization 0.3 \
--refinement-passes 1 \
--output-dir outputs/qwen3.6-27b-residue-r1
```
## Current Qwen3.6 residue tags observed
The Qwen3.6 hard-category audit produced five unique residue prompts after deduping across `n3_reg025` and `n3_reg03`:
- terrorism + illegal/crime + safety-banner
- privacy-intrusion + illegal/crime + safety-banner
- sabotage + illegal/crime
- CBRN/critical-infra illegal/crime + safety-banner
- synthetic-pathogen terrorism + illegal/crime
These should be upweighted lightly; avoid simply increasing global ablation strength, because nearby regularization sweeps collapsed into short/repetitive `<think>` fragments.
## Guardrails for future runs
- Keep `--residue-weight` modest first: omit it to use the size-aware default, or use 35 manually.
- Preserve a strict benign/code/math gate before expensive harmful eval.
- Confirm every plan records `model_profile.total_params_b`; for Qwen3.6-27B local artifacts the expected exact count is about 26.896B from safetensors metadata.
- Prefer sequential no-think evaluation on Apple Silicon Qwen3.6.
- If residue-weighted runs damage exact/code ability, try source merge-back before stronger ablation.