mirror of
https://github.com/elder-plinius/OBLITERATUS.git
synced 2026-08-17 16:37:30 +02:00
163 lines
6.4 KiB
Markdown
163 lines
6.4 KiB
Markdown
# Contributing to OBLITERATUS
|
|
|
|
Thanks for your interest in contributing. This document covers everything you need to get started.
|
|
|
|
## Development Setup
|
|
|
|
```bash
|
|
git clone https://github.com/elder-plinius/OBLITERATUS.git
|
|
cd OBLITERATUS
|
|
python -m pip install --index-url https://download.pytorch.org/whl/cpu "torch>=2.0"
|
|
python - <<'PY' > /tmp/torch-cpu-constraint.txt
|
|
import torch
|
|
print(f"torch=={torch.__version__}")
|
|
PY
|
|
python -m pip install -e ".[dev]" -c /tmp/torch-cpu-constraint.txt
|
|
```
|
|
|
|
This installs CPU PyTorch first, then installs the package in editable mode with pinned development tools while constraining PyTorch to the already-installed CPU build.
|
|
|
|
## Running Tests
|
|
|
|
```bash
|
|
python -m pytest # full suite with coverage
|
|
python -m pytest tests/test_abliterate.py # single file
|
|
python -m pytest -x # stop on first failure
|
|
python -m pytest -k "test_name" # run specific test
|
|
python - <<'PY'
|
|
import obliteratus
|
|
print(getattr(obliteratus, "__version__", "import ok"))
|
|
PY
|
|
python -m obliteratus --help
|
|
```
|
|
|
|
All tests must pass before submitting a PR. Tests are designed to run on CPU without downloading models.
|
|
The mandatory gate currently requires at least 60% repository statement coverage,
|
|
42% branch coverage, 90% coverage of changed executable lines, and 70% statement
|
|
coverage in the device, loader, architecture-profile, CLI, simulated MLX,
|
|
evaluation-metric, reporting, community-contribution, and telemetry boundary
|
|
modules. New changes should raise these floors rather than consume the existing
|
|
margin. A separately measured mature CPU-only scope must remain at or above 80%
|
|
statement and 75% branch coverage; its environment-bound exclusions and their
|
|
conditional-test ownership are versioned in `ci/test-quality-policy.json`.
|
|
|
|
The quality-depth job repeats the highest-consequence pure tests three times in
|
|
different file orders and with different deterministic hash seeds. It also runs
|
|
selective mutation testing over configuration parsing and the coverage-policy
|
|
gate. The initial mutation floor is 70%. Run these checks locally with:
|
|
|
|
```bash
|
|
uv sync --locked --extra dev --group quality
|
|
python scripts/run_repeat_gate.py --output test-results/repeat-gate.json
|
|
mutmut run --max-children 4
|
|
mutmut export-cicd-stats
|
|
python scripts/check_mutation_score.py mutants/mutmut-cicd-stats.json --minimum 70
|
|
```
|
|
|
|
On the calibration runner, the repeat gate took approximately 23 seconds and
|
|
the 141-mutant selection took approximately 11 seconds after environment setup.
|
|
CI retains repeat timing/output, mutation timing, test selection, score, and
|
|
survivor evidence for 14 days. Thresholds may move downward only through a
|
|
time-bounded exception linked to a reviewed repository issue.
|
|
|
|
Hardware, model-download, network-service, operator-UI, and remote-provider tests
|
|
run separately so the pull-request baseline stays offline and credential-free. See
|
|
[`docs/conditional-testing.md`](docs/conditional-testing.md) for manual commands,
|
|
runner labels, pinned resources, credential handling, costs, cadence, and evidence
|
|
freshness. Validate the mapping between those gates and CPU coverage exclusions with:
|
|
|
|
```bash
|
|
python scripts/check_conditional_policy.py
|
|
```
|
|
|
|
## Code Style
|
|
|
|
We use [ruff](https://docs.astral.sh/ruff/) for linting and formatting:
|
|
|
|
```bash
|
|
python -m ruff check --select F app.py obliteratus tests scripts
|
|
python -m ruff check --select E501 --statistics app.py obliteratus tests scripts # known non-blocking line-length debt report
|
|
```
|
|
|
|
- Line length: 100 characters
|
|
- Target: Python 3.10+
|
|
- The CI Ruff gate enforces all Ruff F rules. E501 line-length findings are reported as known non-blocking legacy debt until the baseline is cleaned up.
|
|
- Follow existing patterns in the codebase
|
|
|
|
## Submitting Changes
|
|
|
|
1. Fork the repo and create a branch from `main`
|
|
2. Make your changes
|
|
3. Add or update tests as needed
|
|
4. Run `python -m pytest`, `python -m build --sdist --wheel`, the import/CLI smoke checks, and the CI Ruff gate
|
|
5. Write a clear commit message explaining *why*, not just *what*
|
|
6. Open a pull request
|
|
|
|
## Pull Request Guidelines
|
|
|
|
- Keep PRs focused -- one feature or fix per PR
|
|
- Include a test plan in the PR description
|
|
- Link related issues with `Fixes #123` or `Closes #123`
|
|
- For new analysis modules, include unit tests with synthetic data (no model downloads)
|
|
- Legacy cleanup PRs may receive missing tests as a one-time maintainer courtesy when the change is already otherwise clean. New changes are expected to include relevant tests and keep the full suite passing.
|
|
|
|
## Contributing Experiment Results
|
|
|
|
Beyond code contributions, you can contribute abliteration experiment results to the community dataset used in the research paper. After running abliteration on any model:
|
|
|
|
```bash
|
|
obliteratus obliterate <model> --method advanced --contribute \
|
|
--contribute-notes "Hardware: A100, prompt set: default"
|
|
```
|
|
|
|
This saves a structured JSON file to `community_results/`. To submit your results:
|
|
|
|
1. Run abliteration with `--contribute` on any model/method combination
|
|
2. Open a PR adding your `community_results/*.json` file(s)
|
|
3. The aggregation pipeline will incorporate your data into the paper tables
|
|
|
|
You can preview aggregated results locally:
|
|
|
|
```bash
|
|
obliteratus aggregate --format summary
|
|
obliteratus aggregate --format latex --min-runs 3
|
|
```
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
obliteratus/
|
|
abliterate.py # Core abliteration pipeline
|
|
informed_pipeline.py # Analysis-informed pipeline
|
|
community.py # Community contribution system
|
|
cli.py # CLI entry point
|
|
config.py # YAML config loading
|
|
interactive.py # Interactive mode
|
|
presets.py # Model presets (47 models)
|
|
runner.py # Ablation study runner
|
|
analysis/ # 15 analysis modules
|
|
evaluation/ # Metrics and benchmarks
|
|
models/ # Model loading utilities
|
|
reporting/ # Report generation
|
|
strategies/ # Ablation strategies (layer, head, FFN, embedding)
|
|
tests/ # 44 test files
|
|
paper/ # LaTeX paper
|
|
examples/ # YAML config examples
|
|
```
|
|
|
|
## Reporting Bugs
|
|
|
|
Open an issue with:
|
|
- What you expected to happen
|
|
- What actually happened
|
|
- Steps to reproduce
|
|
- Model name and hardware (GPU/CPU, VRAM)
|
|
|
|
## Security Issues
|
|
|
|
See [SECURITY.md](SECURITY.md) for responsible disclosure of security vulnerabilities.
|
|
|
|
## License
|
|
|
|
By contributing, you agree that your contributions will be licensed under the [AGPL-3.0](LICENSE).
|