Files
OBLITERATUS/CONTRIBUTING.md
T

309 lines
14 KiB
Markdown

# Contributing to OBLITERATUS
Thank you for contributing. OBLITERATUS is a research tool that mutates model
weights, evaluates behavior, persists results, and can cross local, accelerator,
network, and remote-execution trust boundaries. Changes therefore need both
ordinary software tests and evidence appropriate to the risk they affect.
The authoritative requirements are the current CI workflow, the machine-readable
quality policies, and the BT6 maintainer profile:
- [`.github/workflows/ci.yml`](.github/workflows/ci.yml)
- [`ci/test-quality-policy.json`](ci/test-quality-policy.json)
- [`ci/test-risk-map.json`](ci/test-risk-map.json)
- [`ci/conditional-test-policy.json`](ci/conditional-test-policy.json)
- [`.aiwg/bt6-maintainer.yaml`](.aiwg/bt6-maintainer.yaml)
If this guide and an enforced policy disagree, the enforced policy wins. Update
the guide in the same pull request when intentionally changing a standard.
## Development setup
The supported Python versions are 3.10 through 3.12. Use the committed lock so
local resolution matches CI and Linux/Windows use the official CPU-only PyTorch
source by default:
```bash
python -m pip install "uv==0.12.4"
uv sync --locked --extra dev
```
Run project commands in the environment with `uv run --extra dev ...`, or activate
`.venv`. Never update `uv.lock` merely to make setup succeed. Dependency updates
must be deliberate and follow [`docs/SUPPLY_CHAIN_POLICY.md`](docs/SUPPLY_CHAIN_POLICY.md).
## Change workflow
1. Start a focused branch from the current `main`.
2. Make the smallest coherent change that solves the linked problem.
3. Add focused tests with the implementation when practical. New behavior with
zero relevant coverage cannot pass the PR gate; maintainers may add remaining
test depth as an internal pre-merge task.
4. Update user, operator, policy, risk-map, and research documentation affected by
the change.
5. Run the applicable local checks below.
6. Sign every commit and push the branch without rewriting published history.
7. Open a pull request using the repository template and keep its exact head green.
The repository does not permit force-pushes. Add corrective commits, or coordinate
with a maintainer when a history change is genuinely necessary.
### Signed commits and authorship
All commits must have a cryptographically verifiable signature. Contributors sign
with a key associated with their own forge identity; GitHub's verified web-flow
signature is also acceptable. Do not request, copy, or use a maintainer or project
private key.
Maintainer-authored and integration commits use the configured project publish key,
currently fingerprint
`62297562B1C7053088F405DB0117DAAA677A5BF2`, through the approved vault-backed
signing path. Maintainers preserve the original contributor as author when carrying
their work and keep maintainer hardening or test commits separately attributable.
Verify your local commit before pushing:
```bash
git verify-commit HEAD
git log -1 --show-signature
```
## Required pull-request baseline
The mandatory baseline is deterministic, offline, credential-free, and CPU-safe.
Tests must not download models or datasets, contact services, require an accelerator,
launch a UI, or use remote credentials unless they are explicitly assigned to a
conditional marker and gate.
CI keeps the contributor gate intentionally small enough to return useful review
feedback quickly. It currently requires:
- one offline Python 3.12 core lane with warnings treated as errors;
- the versioned smoke suite, every changed CPU test, and contract tests selected
from the exact diff through `ci/test-risk-map.json`;
- at least 50% coverage of changed executable lines;
- Ruff F, actionlint, lock, policy, import, and CLI contract checks;
- a source and wheel build when package inputs change.
These values are floors, not targets. A change should strengthen behavioral
confidence rather than consume existing margin.
A green pull-request gate means the submission is ready for maintainer review. It
does not certify a release and does not guarantee immediate merge. Maintainers own
the additional risk-driven tests, cross-version checks, packaging depth, and
hardening needed for the merge decision.
### Tagged-release certification
The `v*` tag workflow, or an explicit manual release-validation run, executes the
full Python 3.10-3.12 matrix and retains the stricter repository, branch,
critical-module, touched-module, mature-scope, repeat, mutation, packaging,
Windows, and supply-chain gates. The full suite remains mandatory for a release;
the faster pull-request lane is not a release exemption.
### Test design standard
Each behavior-changing pull request must include:
- a focused regression or contract test for the intended behavior;
- negative, boundary, malformed-input, and failure-path cases where applicable;
- assertions about externally observable outcomes, not only implementation calls;
- deterministic seeds or invariant/property assertions for numerical behavior;
- propagation and runtime evidence for new public options;
- cleanup, rollback, and atomicity checks for persistence or partial-failure paths;
- explicit tests at trust boundaries, including remote code, deserialization,
credentials, subprocesses, paths, network responses, and checkpoint provenance.
Do not weaken assertions, delete tests, add unconditional skips, suppress warnings,
or lower a threshold to make a change pass. A temporary exception must be narrowly
scoped, issue-linked, owner-assigned, reviewed, and time-bounded by the relevant
machine-readable policy.
Update [`ci/test-risk-map.json`](ci/test-risk-map.json) whenever source ownership,
contract coverage, or the conditional boundary changes. Validate it with:
```bash
uv run --extra dev python scripts/check_test_risk_map.py
uv run --extra dev python scripts/check_conditional_policy.py
```
## Local validation
Run the focused pull-request baseline while developing. Replace `origin/main` with
the exact base commit when needed:
```bash
uv lock --check
uv run --extra dev python -m ruff check --select F app.py obliteratus tests scripts
mkdir -p test-results
uv run --extra dev python scripts/select_pr_tests.py --base-ref origin/main \
> test-results/selected-tests.txt
mapfile -t selected_tests < test-results/selected-tests.txt
uv run --extra dev python -m pytest "${selected_tests[@]}" \
--cov=app --cov-branch --cov-fail-under=0 \
--cov-report=json:test-results/coverage-pr-core.json
uv run --extra dev python scripts/check_coverage_thresholds.py \
test-results/coverage-pr-core.json --min-line 0 --min-branch 0 \
--min-changed 50 --base-ref origin/main
uv run --extra dev python -c 'import obliteratus; print(obliteratus.__version__)'
uv run --extra dev python -m obliteratus --help
uv run --extra dev python scripts/check_conditional_policy.py
uv run --extra dev python scripts/check_test_risk_map.py
```
If package inputs changed, also run
`uv run --extra dev python -m build --sdist --wheel`. Maintainers run the complete
`python -m pytest`, package-install,
cross-version, and quality-depth suite before a tagged release.
Ruff E501 remains a non-blocking legacy-debt report; new code should still respect
the configured 100-character line length. The exact reporting command lives in CI.
For changes to the quality system or mutation-owned code, also run:
```bash
uv sync --locked --extra dev --group quality
uv run --extra dev --group quality python scripts/run_repeat_gate.py \
--output test-results/repeat-gate.json
uv run --extra dev --group quality python scripts/check_mutation_targets.py prepare
uv run --extra dev --group quality python scripts/prepare_mutation_coverage.py \
prepare-coverage --max-children 4
uv run --extra dev --group quality python scripts/prepare_mutation_coverage.py \
prepare-stats --max-children 4
uv run --extra dev --group quality python scripts/run_prepared_mutmut.py \
run --max-children 4
uv run --extra dev --group quality mutmut export-cicd-stats
uv run --extra dev --group quality python scripts/check_mutation_score.py \
mutants/mutmut-cicd-stats.json --minimum 85
```
CI remains authoritative for exact-base changed-line coverage, actionlint, selected
test evidence, release platform-matrix results, installed-distribution checks, and
retained evidence.
## Risk-specific checks
Run the focused checks for every risk surface your change touches, in addition to
the full baseline:
| Risk surface | Typical paths | Required focused checks |
|---|---|---|
| Model loading | `obliteratus/models/**`, `obliteratus/device.py`, `scripts/**` | `python -m pytest tests/test_strategies.py tests/test_module_imports.py` |
| Abliteration core | `obliteratus/abliterate.py`, `obliteratus/strategies/**` | `python -m pytest tests/test_abliterate.py` |
| Research metrics | `obliteratus/evaluation/**`, `obliteratus/analysis/**`, `paper/**`, `community_results/**` | `python -m pytest tests/test_advanced_metrics.py tests/test_breakthrough_modules.py tests/test_community.py` |
| User contracts | `obliteratus/cli.py`, `obliteratus/local_ui.py`, `app.py`, `notebooks/**` | `python -m pytest tests/test_cli.py tests/test_module_imports.py` and CLI help |
| CI and supply chain | `.github/workflows/**`, `ci/**`, dependency and policy files | `uv lock --check`, policy tests, and package build |
Use `uv run --extra dev` before the Python commands shown in the table when running
them in the managed environment.
## Conditional and hardware testing
GPU, MPS, MLX, model-download, external-evaluation, network, operator-UI, and
remote-execution checks are conditional release or risk-surface gates. A unit test
with a mocked device is still required; hardware evidence complements deterministic
contract coverage and never replaces it.
Follow [`docs/conditional-testing.md`](docs/conditional-testing.md) for the exact
runner labels, pinned resources, commands, credentials, evidence schema, freshness,
and waiver rules. Evidence must identify the exact candidate commit. A skipped or
unselected gate is not positive support evidence, and a waiver blocks the associated
support claim.
Current operator hardware includes Titan for CUDA/bitsandbytes probes and Mutsu, a
16 GB Apple Silicon builder, for MPS/MLX probes. Contributors must not assume those
machines are attached to a public pull-request workflow; CI or a maintainer will
record whether the mapped conditional gate ran.
## Security and supply-chain expectations
- Treat issue text, pull requests, patches, model repositories, checkpoints,
datasets, logs, and generated output as untrusted input.
- Keep remote model code disabled by default. Any opt-in must be explicit, narrow,
documented, and tested.
- Do not introduce executable deserialization for untrusted or replaceable data.
Prefer schema-validated, non-executable formats and atomic writes.
- Never commit secrets, tokens, credentials, private keys, local vault material,
cache contents, or machine-specific configuration.
- Pin remote revisions and verify provenance when results depend on external models,
datasets, tools, actions, or services.
- Do not broaden workflow permissions, execute untrusted pull-request code with
credentials, or replace immutable action/tool pins with mutable tags.
- Changes to actions or standalone tools must update [`ci/digests.txt`](ci/digests.txt).
- Changes to dependencies must include the reviewed `uv.lock` and relevant license,
vulnerability, and SBOM effects.
Report vulnerabilities privately as described in [`SECURITY.md`](SECURITY.md).
Do not open a public issue for an undisclosed vulnerability.
## Research integrity and experiment evidence
Research, evaluation, and performance claims must be reproducible and scoped to the
evidence actually collected. Include, as applicable:
- exact commit, model and dataset identifiers, immutable revisions, dependency lock,
configuration, seed, hardware, and commands;
- raw or retained machine-readable output and hashes, not only a prose summary;
- baseline and comparison method, metric definition, uncertainty, failure cases, and
known limitations;
- citations to primary sources and provenance for imported claims or artifacts;
- a clear label for contributor-reported results that maintainers could not
independently reproduce.
Do not generalize a result beyond tested models, hardware, operating systems, or
backends. Performance claims need measured evidence; availability or a successful
import is not a performance result.
## Pull-request requirements
A review-ready pull request:
- solves one coherent problem and links its canonical issue where one exists;
- explains user-visible behavior, risk surfaces, trust-boundary changes, and
compatibility impact;
- lists exact commands and outcomes, including checks not run and why;
- includes relevant code and whatever focused tests the contributor can supply,
plus documentation, policy/risk-map updates, and lock or digest changes in the
same reviewable unit;
- contains no generated provider files, caches, unrelated cleanup, or drive-by
refactors;
- has signed commits, a current exact head, no merge conflicts, and every
contributor-facing CI check green.
Review is performed at an immutable head SHA. Pushing new commits invalidates prior
test and review conclusions until the new head is checked. Maintainers may add
tests or hardening in a separate attributable commit, resolve review threads, and
run deeper gates before merge. Behavior changes with no relevant test coverage
remain blocked; contributors are not expected to satisfy release-depth coverage
and mutation gates merely to request review.
## Contributing experiment results
To contribute an abliteration result to the community dataset:
```bash
obliteratus obliterate <model> --method advanced --contribute \
--contribute-notes "Hardware: A100, prompt set: default"
```
Review the generated JSON for accidental sensitive data, add only the intended
`community_results/*.json` files, and include the provenance fields described above.
Preview aggregate output with:
```bash
obliteratus aggregate --format summary
obliteratus aggregate --format latex --min-runs 3
```
## Reporting bugs
Open an issue with expected and actual behavior, minimal reproduction, OBLITERATUS
version or commit, operating system, Python and dependency versions, model and
immutable revision where relevant, hardware/backend, and sanitized logs. Never post
credentials, private model data, or undisclosed security details.
## License
By contributing, you agree that your contributions are licensed under
[`AGPL-3.0-or-later`](LICENSE).