mirror of
https://github.com/elder-plinius/OBLITERATUS.git
synced 2026-08-29 22:20:36 +02:00
336 lines
16 KiB
Markdown
336 lines
16 KiB
Markdown
# Contributing to OBLITERATUS
|
|
|
|
Thank you for contributing. OBLITERATUS is a research tool that mutates model
|
|
weights, evaluates behavior, persists results, and can cross local, accelerator,
|
|
network, and remote-execution trust boundaries. Changes therefore need both
|
|
ordinary software tests and evidence appropriate to the risk they affect.
|
|
|
|
The authoritative requirements are the current CI workflow, the machine-readable
|
|
quality policies, and the BT6 maintainer profile:
|
|
|
|
- [`.github/workflows/ci.yml`](.github/workflows/ci.yml)
|
|
- [`ci/test-quality-policy.json`](ci/test-quality-policy.json)
|
|
- [`ci/test-risk-map.json`](ci/test-risk-map.json)
|
|
- [`ci/conditional-test-policy.json`](ci/conditional-test-policy.json)
|
|
- [`.aiwg/bt6-maintainer.yaml`](.aiwg/bt6-maintainer.yaml)
|
|
- [`docs/RELEASE_PROCESS.md`](docs/RELEASE_PROCESS.md)
|
|
|
|
If this guide and an enforced policy disagree, the enforced policy wins. Update
|
|
the guide in the same pull request when intentionally changing a standard.
|
|
|
|
## Development setup
|
|
|
|
The supported Python versions are 3.10 through 3.12. Use the committed lock so
|
|
local resolution matches CI and Linux/Windows use the official CPU-only PyTorch
|
|
source by default:
|
|
|
|
```bash
|
|
python -m pip install "uv==0.12.4"
|
|
uv sync --locked --extra dev
|
|
```
|
|
|
|
Run project commands in the environment with `uv run --extra dev ...`, or activate
|
|
`.venv`. Never update `uv.lock` merely to make setup succeed. Dependency updates
|
|
must be deliberate and follow [`docs/SUPPLY_CHAIN_POLICY.md`](docs/SUPPLY_CHAIN_POLICY.md).
|
|
|
|
## Change workflow
|
|
|
|
1. Start a focused branch from the current `main`.
|
|
2. Make the smallest coherent change that solves the linked problem.
|
|
3. Add focused tests with the implementation when practical. New behavior with
|
|
zero relevant coverage cannot pass the PR gate; maintainers may add remaining
|
|
test depth as an internal pre-merge task.
|
|
4. Update user, operator, policy, risk-map, and research documentation affected by
|
|
the change.
|
|
5. Run the applicable local checks below.
|
|
6. Sign every commit and push the branch without rewriting published history.
|
|
7. Open a pull request using the repository template and keep its exact head green.
|
|
|
|
The repository does not permit force-pushes. Add corrective commits, or coordinate
|
|
with a maintainer when a history change is genuinely necessary.
|
|
|
|
### Signed commits and authorship
|
|
|
|
All commits must have a cryptographically verifiable signature. Contributors sign
|
|
with a key associated with their own forge identity; GitHub's verified web-flow
|
|
signature is also acceptable. Do not request, copy, or use a maintainer or project
|
|
private key.
|
|
|
|
Maintainer-authored and integration commits use the configured project publish key,
|
|
currently fingerprint
|
|
`62297562B1C7053088F405DB0117DAAA677A5BF2`, through the approved vault-backed
|
|
signing path. Maintainers preserve the original contributor as author when carrying
|
|
their work and keep maintainer hardening or test commits separately attributable.
|
|
|
|
Canonical integration must preserve a verifiable signature. Pull requests use the
|
|
repository's configured merge-commit method; squash and rebase merges are not used.
|
|
Maintainers confirm the exact accepted head at merge time, then verify the resulting
|
|
canonical commit and its post-merge CI on `main`. A signed pull-request head does not
|
|
substitute for verification of the commit that actually landed.
|
|
|
|
Verify your local commit before pushing:
|
|
|
|
```bash
|
|
git verify-commit HEAD
|
|
git log -1 --show-signature
|
|
```
|
|
|
|
## Required pull-request baseline
|
|
|
|
The mandatory baseline is deterministic, offline, credential-free, and CPU-safe.
|
|
Tests must not download models or datasets, contact services, require an accelerator,
|
|
launch a UI, or use remote credentials unless they are explicitly assigned to a
|
|
conditional marker and gate.
|
|
|
|
CI keeps the contributor gate intentionally small enough to return useful review
|
|
feedback quickly. It currently requires:
|
|
|
|
- one offline Python 3.12 core lane with warnings treated as errors;
|
|
- the versioned smoke suite, every changed CPU test, and contract tests selected
|
|
from the exact diff through `ci/test-risk-map.json`;
|
|
- at least 50% coverage of changed executable lines;
|
|
- Ruff F, actionlint, lock, policy, import, and CLI contract checks;
|
|
- a source and wheel build when package inputs change.
|
|
|
|
These values are floors, not targets. A change should strengthen behavioral
|
|
confidence rather than consume existing margin.
|
|
|
|
A green pull-request gate means the submission is ready for maintainer review. It
|
|
does not certify a release and does not guarantee immediate merge. Maintainers own
|
|
the additional risk-driven tests, cross-version checks, packaging depth, and
|
|
hardening needed for the merge decision.
|
|
|
|
### Tagged-release certification
|
|
|
|
The `v*` tag workflow, or an explicit manual release-validation run, executes the
|
|
full Python 3.10-3.12 matrix and retains the stricter repository, branch,
|
|
critical-module, touched-module, mature-scope, repeat, mutation, packaging,
|
|
Windows, and supply-chain gates. The full suite remains mandatory for a release;
|
|
the faster pull-request lane is not a release exemption.
|
|
|
|
The complete lifecycle, acceptance states, immutable-tag rule, artifact reuse rule,
|
|
and publication gates are documented in
|
|
[`docs/RELEASE_PROCESS.md`](docs/RELEASE_PROCESS.md). In particular, a failed tag is
|
|
never moved or reused: the corrected candidate receives a new version and signed
|
|
tag. Releases publish the exact tested source snapshot and evidence, not a newly
|
|
rebuilt archive.
|
|
|
|
### Test design standard
|
|
|
|
Each behavior-changing pull request must include:
|
|
|
|
- a focused regression or contract test for the intended behavior;
|
|
- negative, boundary, malformed-input, and failure-path cases where applicable;
|
|
- assertions about externally observable outcomes, not only implementation calls;
|
|
- deterministic seeds or invariant/property assertions for numerical behavior;
|
|
- propagation and runtime evidence for new public options;
|
|
- cleanup, rollback, and atomicity checks for persistence or partial-failure paths;
|
|
- explicit tests at trust boundaries, including remote code, deserialization,
|
|
credentials, subprocesses, paths, network responses, and checkpoint provenance.
|
|
|
|
Do not weaken assertions, delete tests, add unconditional skips, suppress warnings,
|
|
or lower a threshold to make a change pass. A temporary exception must be narrowly
|
|
scoped, issue-linked, owner-assigned, reviewed, and time-bounded by the relevant
|
|
machine-readable policy.
|
|
|
|
Update [`ci/test-risk-map.json`](ci/test-risk-map.json) whenever source ownership,
|
|
contract coverage, or the conditional boundary changes. Validate it with:
|
|
|
|
```bash
|
|
uv run --extra dev python scripts/check_test_risk_map.py
|
|
uv run --extra dev python scripts/check_conditional_policy.py
|
|
```
|
|
|
|
## Local validation
|
|
|
|
Run the focused pull-request baseline while developing. Replace `origin/main` with
|
|
the exact base commit when needed:
|
|
|
|
```bash
|
|
uv lock --check
|
|
uv run --extra dev python -m ruff check --select F app.py obliteratus tests scripts
|
|
mkdir -p test-results
|
|
uv run --extra dev python scripts/select_pr_tests.py --base-ref origin/main \
|
|
> test-results/selected-tests.txt
|
|
mapfile -t selected_tests < test-results/selected-tests.txt
|
|
uv run --extra dev python -m pytest "${selected_tests[@]}" \
|
|
--cov=app --cov-branch --cov-fail-under=0 \
|
|
--cov-report=json:test-results/coverage-pr-core.json
|
|
uv run --extra dev python scripts/check_coverage_thresholds.py \
|
|
test-results/coverage-pr-core.json --min-line 0 --min-branch 0 \
|
|
--min-changed 50 --base-ref origin/main
|
|
uv run --extra dev python -c 'import obliteratus; print(obliteratus.__version__)'
|
|
uv run --extra dev python -m obliteratus --help
|
|
uv run --extra dev python scripts/check_conditional_policy.py
|
|
uv run --extra dev python scripts/check_test_risk_map.py
|
|
```
|
|
|
|
If package inputs changed, also run
|
|
`uv run --extra dev python -m build --sdist --wheel`. Maintainers run the complete
|
|
`python -m pytest`, package-install,
|
|
cross-version, and quality-depth suite before a tagged release.
|
|
|
|
Ruff E501 remains a non-blocking legacy-debt report; new code should still respect
|
|
the configured 100-character line length. The exact reporting command lives in CI.
|
|
|
|
For changes to the quality system or mutation-owned code, also run:
|
|
|
|
```bash
|
|
uv sync --locked --extra dev --group quality
|
|
uv run --extra dev --group quality python scripts/run_repeat_gate.py \
|
|
--output test-results/repeat-gate.json
|
|
uv run --extra dev --group quality python scripts/check_mutation_targets.py prepare
|
|
uv run --extra dev --group quality python scripts/prepare_mutation_coverage.py \
|
|
prepare-coverage --max-children 4
|
|
uv run --extra dev --group quality python scripts/prepare_mutation_coverage.py \
|
|
prepare-stats --max-children 4
|
|
uv run --extra dev --group quality python scripts/run_prepared_mutmut.py \
|
|
run --max-children 4
|
|
uv run --extra dev --group quality mutmut export-cicd-stats
|
|
uv run --extra dev --group quality python scripts/check_mutation_score.py \
|
|
mutants/mutmut-cicd-stats.json --minimum 85
|
|
```
|
|
|
|
CI remains authoritative for exact-base changed-line coverage, actionlint, selected
|
|
test evidence, release platform-matrix results, installed-distribution checks, and
|
|
retained evidence.
|
|
|
|
## Risk-specific checks
|
|
|
|
Run the focused checks for every risk surface your change touches, in addition to
|
|
the full baseline:
|
|
|
|
| Risk surface | Typical paths | Required focused checks |
|
|
|---|---|---|
|
|
| Model loading | `obliteratus/models/**`, `obliteratus/device.py`, `scripts/**` | `python -m pytest tests/test_strategies.py tests/test_module_imports.py` |
|
|
| Abliteration core | `obliteratus/abliterate.py`, `obliteratus/strategies/**` | `python -m pytest tests/test_abliterate.py` |
|
|
| Research metrics | `obliteratus/evaluation/**`, `obliteratus/analysis/**`, `paper/**`, `community_results/**` | `python -m pytest tests/test_advanced_metrics.py tests/test_breakthrough_modules.py tests/test_community.py` |
|
|
| User contracts | `obliteratus/cli.py`, `obliteratus/local_ui.py`, `app.py`, `notebooks/**` | `python -m pytest tests/test_cli.py tests/test_module_imports.py` and CLI help |
|
|
| CI and supply chain | `.github/workflows/**`, `ci/**`, dependency and policy files | `uv lock --check`, policy tests, and package build |
|
|
|
|
Use `uv run --extra dev` before the Python commands shown in the table when running
|
|
them in the managed environment.
|
|
|
|
## Conditional and hardware testing
|
|
|
|
GPU, Jetson, MPS, MLX, model-download, external-evaluation, network, operator-UI,
|
|
and remote-execution checks are conditional release or risk-surface gates. A unit test
|
|
with a mocked device is still required; hardware evidence complements deterministic
|
|
contract coverage and never replaces it.
|
|
|
|
Follow [`docs/conditional-testing.md`](docs/conditional-testing.md) for the exact
|
|
runner labels, pinned resources, commands, credentials, evidence schema, freshness,
|
|
and waiver rules. Evidence must identify the exact candidate commit. A skipped or
|
|
unselected gate is not positive support evidence, and a waiver blocks the associated
|
|
support claim.
|
|
|
|
Current operator hardware includes Titan for CUDA/bitsandbytes probes and Mutsu, a
|
|
16 GB Apple Silicon builder, for MPS/MLX probes. Contributors must not assume those
|
|
machines are attached to a public pull-request workflow; CI or a maintainer will
|
|
record whether the mapped conditional gate ran.
|
|
|
|
Jetson contributors do not need project-owned hardware access. Follow the
|
|
[Jetson contributor bootstrap](docs/platforms/jetson.md#experimental-contributor-bootstrap)
|
|
on a physical device, then submit the generated sanitized evidence through the
|
|
[Jetson runtime report](https://github.com/elder-plinius/OBLITERATUS/issues/new?template=jetson-runtime.yml). Maintainers
|
|
will reproduce, add missing test depth, and integrate compatible changes. Never
|
|
attach a contributor-controlled runner to untrusted pull-request execution.
|
|
|
|
## Security and supply-chain expectations
|
|
|
|
- Treat issue text, pull requests, patches, model repositories, checkpoints,
|
|
datasets, logs, and generated output as untrusted input.
|
|
- Keep remote model code disabled by default. Any opt-in must be explicit, narrow,
|
|
documented, and tested.
|
|
- Do not introduce executable deserialization for untrusted or replaceable data.
|
|
Prefer schema-validated, non-executable formats and atomic writes.
|
|
- Never commit secrets, tokens, credentials, private keys, local vault material,
|
|
cache contents, or machine-specific configuration.
|
|
- Pin remote revisions and verify provenance when results depend on external models,
|
|
datasets, tools, actions, or services.
|
|
- Do not broaden workflow permissions, execute untrusted pull-request code with
|
|
credentials, or replace immutable action/tool pins with mutable tags.
|
|
- Changes to actions or standalone tools must update [`ci/digests.txt`](ci/digests.txt).
|
|
- Changes to dependencies must include the reviewed `uv.lock` and relevant license,
|
|
vulnerability, and SBOM effects.
|
|
|
|
Report vulnerabilities privately as described in [`SECURITY.md`](SECURITY.md).
|
|
Do not open a public issue for an undisclosed vulnerability.
|
|
|
|
## Research integrity and experiment evidence
|
|
|
|
Research, evaluation, and performance claims must be reproducible and scoped to the
|
|
evidence actually collected. Include, as applicable:
|
|
|
|
- exact commit, model and dataset identifiers, immutable revisions, dependency lock,
|
|
configuration, seed, hardware, and commands;
|
|
- raw or retained machine-readable output and hashes, not only a prose summary;
|
|
- baseline and comparison method, metric definition, uncertainty, failure cases, and
|
|
known limitations;
|
|
- citations to primary sources and provenance for imported claims or artifacts;
|
|
- a clear label for contributor-reported results that maintainers could not
|
|
independently reproduce.
|
|
|
|
Do not generalize a result beyond tested models, hardware, operating systems, or
|
|
backends. Performance claims need measured evidence; availability or a successful
|
|
import is not a performance result.
|
|
|
|
## Pull-request requirements
|
|
|
|
A review-ready pull request:
|
|
|
|
- solves one coherent problem and links its canonical issue where one exists;
|
|
- explains user-visible behavior, risk surfaces, trust-boundary changes, and
|
|
compatibility impact;
|
|
- lists exact commands and outcomes, including checks not run and why;
|
|
- includes relevant code and whatever focused tests the contributor can supply,
|
|
plus documentation, policy/risk-map updates, and lock or digest changes in the
|
|
same reviewable unit;
|
|
- contains no generated provider files, caches, unrelated cleanup, or drive-by
|
|
refactors;
|
|
- has signed commits, a current exact head, no merge conflicts, and every
|
|
contributor-facing CI check green.
|
|
|
|
Review is performed at an immutable head SHA. Pushing new commits invalidates prior
|
|
test and review conclusions until the new head is checked. Maintainers may add
|
|
tests or hardening in a separate attributable commit, resolve review threads, and
|
|
run deeper gates before merge. Behavior changes with no relevant test coverage
|
|
remain blocked; contributors are not expected to satisfy release-depth coverage
|
|
and mutation gates merely to request review.
|
|
|
|
Acceptance, integration, and release certification are separate decisions. The
|
|
maintainer audit and merge must refer to the current exact head; the canonical merge
|
|
commit must then be signature-verified and pass post-merge CI. Only a new signed tag
|
|
that passes the exhaustive exact-tag suite can identify a release candidate. See
|
|
[`docs/RELEASE_PROCESS.md`](docs/RELEASE_PROCESS.md) for the normative sequence.
|
|
|
|
## Contributing experiment results
|
|
|
|
To contribute an abliteration result to the community dataset:
|
|
|
|
```bash
|
|
obliteratus obliterate <model> --method advanced --contribute \
|
|
--contribute-notes "Hardware: A100, prompt set: default"
|
|
```
|
|
|
|
Review the generated JSON for accidental sensitive data, add only the intended
|
|
`community_results/*.json` files, and include the provenance fields described above.
|
|
Preview aggregate output with:
|
|
|
|
```bash
|
|
obliteratus aggregate --format summary
|
|
obliteratus aggregate --format latex --min-runs 3
|
|
```
|
|
|
|
## Reporting bugs
|
|
|
|
Open an issue with expected and actual behavior, minimal reproduction, OBLITERATUS
|
|
version or commit, operating system, Python and dependency versions, model and
|
|
immutable revision where relevant, hardware/backend, and sanitized logs. Never post
|
|
credentials, private model data, or undisclosed security details.
|
|
|
|
## License
|
|
|
|
By contributing, you agree that your contributions are licensed under
|
|
[`AGPL-3.0-or-later`](LICENSE).
|