mirror of
https://github.com/elder-plinius/OBLITERATUS.git
synced 2026-08-17 16:37:30 +02:00
284 lines
13 KiB
Markdown
284 lines
13 KiB
Markdown
# Contributing to OBLITERATUS
|
|
|
|
Thank you for contributing. OBLITERATUS is a research tool that mutates model
|
|
weights, evaluates behavior, persists results, and can cross local, accelerator,
|
|
network, and remote-execution trust boundaries. Changes therefore need both
|
|
ordinary software tests and evidence appropriate to the risk they affect.
|
|
|
|
The authoritative requirements are the current CI workflow, the machine-readable
|
|
quality policies, and the BT6 maintainer profile:
|
|
|
|
- [`.github/workflows/ci.yml`](.github/workflows/ci.yml)
|
|
- [`ci/test-quality-policy.json`](ci/test-quality-policy.json)
|
|
- [`ci/test-risk-map.json`](ci/test-risk-map.json)
|
|
- [`ci/conditional-test-policy.json`](ci/conditional-test-policy.json)
|
|
- [`.aiwg/bt6-maintainer.yaml`](.aiwg/bt6-maintainer.yaml)
|
|
|
|
If this guide and an enforced policy disagree, the enforced policy wins. Update
|
|
the guide in the same pull request when intentionally changing a standard.
|
|
|
|
## Development setup
|
|
|
|
The supported Python versions are 3.10 through 3.12. Use the committed lock so
|
|
local resolution matches CI and Linux/Windows use the official CPU-only PyTorch
|
|
source by default:
|
|
|
|
```bash
|
|
python -m pip install "uv==0.12.4"
|
|
uv sync --locked --extra dev
|
|
```
|
|
|
|
Run project commands in the environment with `uv run --extra dev ...`, or activate
|
|
`.venv`. Never update `uv.lock` merely to make setup succeed. Dependency updates
|
|
must be deliberate and follow [`docs/SUPPLY_CHAIN_POLICY.md`](docs/SUPPLY_CHAIN_POLICY.md).
|
|
|
|
## Change workflow
|
|
|
|
1. Start a focused branch from the current `main`.
|
|
2. Make the smallest coherent change that solves the linked problem.
|
|
3. Add tests with the implementation. New behavior without relevant tests is not
|
|
merge-ready.
|
|
4. Update user, operator, policy, risk-map, and research documentation affected by
|
|
the change.
|
|
5. Run the applicable local checks below.
|
|
6. Sign every commit and push the branch without rewriting published history.
|
|
7. Open a pull request using the repository template and keep its exact head green.
|
|
|
|
The repository does not permit force-pushes. Add corrective commits, or coordinate
|
|
with a maintainer when a history change is genuinely necessary.
|
|
|
|
### Signed commits and authorship
|
|
|
|
All commits must have a cryptographically verifiable signature. Contributors sign
|
|
with a key associated with their own forge identity; GitHub's verified web-flow
|
|
signature is also acceptable. Do not request, copy, or use a maintainer or project
|
|
private key.
|
|
|
|
Maintainer-authored and integration commits use the configured project publish key,
|
|
currently fingerprint
|
|
`62297562B1C7053088F405DB0117DAAA677A5BF2`, through the approved vault-backed
|
|
signing path. Maintainers preserve the original contributor as author when carrying
|
|
their work and keep maintainer hardening or test commits separately attributable.
|
|
|
|
Verify your local commit before pushing:
|
|
|
|
```bash
|
|
git verify-commit HEAD
|
|
git log -1 --show-signature
|
|
```
|
|
|
|
## Required pull-request baseline
|
|
|
|
The mandatory baseline is deterministic, offline, credential-free, and CPU-safe.
|
|
Tests must not download models or datasets, contact services, require an accelerator,
|
|
launch a UI, or use remote credentials unless they are explicitly assigned to a
|
|
conditional marker and gate.
|
|
|
|
CI currently requires:
|
|
|
|
- the full test matrix on Python 3.10, 3.11, and 3.12 with warnings treated as
|
|
errors;
|
|
- at least 75% repository line coverage and 60% repository branch coverage;
|
|
- at least 95% coverage of changed executable lines;
|
|
- no independent line or branch regression in a touched production module versus
|
|
the exact base commit;
|
|
- at least 80% line and 75% branch coverage for a new production module;
|
|
- at least 70% line coverage for each policy-designated critical boundary module;
|
|
- at least 94% line and 84% branch coverage for the mature CPU-testable scope;
|
|
- at least an 85% score in the bounded selective mutation campaign;
|
|
- zero unexpected warnings and no unowned, expired, or policy-invalid quarantine;
|
|
- Ruff F, actionlint, package, installed-wheel, installed-sdist, supply-chain, and
|
|
policy checks.
|
|
|
|
These values are floors, not targets. A change should strengthen behavioral
|
|
confidence rather than consume existing margin.
|
|
|
|
### Test design standard
|
|
|
|
Each behavior-changing pull request must include:
|
|
|
|
- a focused regression or contract test for the intended behavior;
|
|
- negative, boundary, malformed-input, and failure-path cases where applicable;
|
|
- assertions about externally observable outcomes, not only implementation calls;
|
|
- deterministic seeds or invariant/property assertions for numerical behavior;
|
|
- propagation and runtime evidence for new public options;
|
|
- cleanup, rollback, and atomicity checks for persistence or partial-failure paths;
|
|
- explicit tests at trust boundaries, including remote code, deserialization,
|
|
credentials, subprocesses, paths, network responses, and checkpoint provenance.
|
|
|
|
Do not weaken assertions, delete tests, add unconditional skips, suppress warnings,
|
|
or lower a threshold to make a change pass. A temporary exception must be narrowly
|
|
scoped, issue-linked, owner-assigned, reviewed, and time-bounded by the relevant
|
|
machine-readable policy.
|
|
|
|
Update [`ci/test-risk-map.json`](ci/test-risk-map.json) whenever source ownership,
|
|
contract coverage, or the conditional boundary changes. Validate it with:
|
|
|
|
```bash
|
|
uv run --extra dev python scripts/check_test_risk_map.py
|
|
uv run --extra dev python scripts/check_conditional_policy.py
|
|
```
|
|
|
|
## Local validation
|
|
|
|
Run focused tests while developing, then the complete baseline before requesting
|
|
review:
|
|
|
|
```bash
|
|
uv lock --check
|
|
uv run --extra dev python -m ruff check --select F app.py obliteratus tests scripts
|
|
uv run --extra dev python -m pytest
|
|
uv run --extra dev python -m build --sdist --wheel
|
|
uv run --extra dev python -c 'import obliteratus; print(obliteratus.__version__)'
|
|
uv run --extra dev python -m obliteratus --help
|
|
uv run --extra dev python scripts/check_conditional_policy.py
|
|
uv run --extra dev python scripts/check_test_risk_map.py
|
|
```
|
|
|
|
Ruff E501 remains a non-blocking legacy-debt report; new code should still respect
|
|
the configured 100-character line length. The exact reporting command lives in CI.
|
|
|
|
For changes to the quality system or mutation-owned code, also run:
|
|
|
|
```bash
|
|
uv sync --locked --extra dev --group quality
|
|
uv run --extra dev --group quality python scripts/run_repeat_gate.py \
|
|
--output test-results/repeat-gate.json
|
|
uv run --extra dev --group quality python scripts/check_mutation_targets.py prepare
|
|
uv run --extra dev --group quality python scripts/prepare_mutation_coverage.py \
|
|
prepare-coverage --max-children 4
|
|
uv run --extra dev --group quality python scripts/prepare_mutation_coverage.py \
|
|
prepare-stats --max-children 4
|
|
uv run --extra dev --group quality python scripts/run_prepared_mutmut.py \
|
|
run --max-children 4
|
|
uv run --extra dev --group quality mutmut export-cicd-stats
|
|
uv run --extra dev --group quality python scripts/check_mutation_score.py \
|
|
mutants/mutmut-cicd-stats.json --minimum 85
|
|
```
|
|
|
|
CI remains authoritative for exact-base coverage comparison, actionlint, platform
|
|
matrix results, installed-distribution checks, and retained evidence.
|
|
|
|
## Risk-specific checks
|
|
|
|
Run the focused checks for every risk surface your change touches, in addition to
|
|
the full baseline:
|
|
|
|
| Risk surface | Typical paths | Required focused checks |
|
|
|---|---|---|
|
|
| Model loading | `obliteratus/models/**`, `obliteratus/device.py`, `scripts/**` | `python -m pytest tests/test_strategies.py tests/test_module_imports.py` |
|
|
| Abliteration core | `obliteratus/abliterate.py`, `obliteratus/strategies/**` | `python -m pytest tests/test_abliterate.py` |
|
|
| Research metrics | `obliteratus/evaluation/**`, `obliteratus/analysis/**`, `paper/**`, `community_results/**` | `python -m pytest tests/test_advanced_metrics.py tests/test_breakthrough_modules.py tests/test_community.py` |
|
|
| User contracts | `obliteratus/cli.py`, `obliteratus/local_ui.py`, `app.py`, `notebooks/**` | `python -m pytest tests/test_cli.py tests/test_module_imports.py` and CLI help |
|
|
| CI and supply chain | `.github/workflows/**`, `ci/**`, dependency and policy files | `uv lock --check`, policy tests, and package build |
|
|
|
|
Use `uv run --extra dev` before the Python commands shown in the table when running
|
|
them in the managed environment.
|
|
|
|
## Conditional and hardware testing
|
|
|
|
GPU, MPS, MLX, model-download, external-evaluation, network, operator-UI, and
|
|
remote-execution checks are conditional release or risk-surface gates. A unit test
|
|
with a mocked device is still required; hardware evidence complements deterministic
|
|
contract coverage and never replaces it.
|
|
|
|
Follow [`docs/conditional-testing.md`](docs/conditional-testing.md) for the exact
|
|
runner labels, pinned resources, commands, credentials, evidence schema, freshness,
|
|
and waiver rules. Evidence must identify the exact candidate commit. A skipped or
|
|
unselected gate is not positive support evidence, and a waiver blocks the associated
|
|
support claim.
|
|
|
|
Current operator hardware includes Titan for CUDA/bitsandbytes probes and Mutsu, a
|
|
16 GB Apple Silicon builder, for MPS/MLX probes. Contributors must not assume those
|
|
machines are attached to a public pull-request workflow; CI or a maintainer will
|
|
record whether the mapped conditional gate ran.
|
|
|
|
## Security and supply-chain expectations
|
|
|
|
- Treat issue text, pull requests, patches, model repositories, checkpoints,
|
|
datasets, logs, and generated output as untrusted input.
|
|
- Keep remote model code disabled by default. Any opt-in must be explicit, narrow,
|
|
documented, and tested.
|
|
- Do not introduce executable deserialization for untrusted or replaceable data.
|
|
Prefer schema-validated, non-executable formats and atomic writes.
|
|
- Never commit secrets, tokens, credentials, private keys, local vault material,
|
|
cache contents, or machine-specific configuration.
|
|
- Pin remote revisions and verify provenance when results depend on external models,
|
|
datasets, tools, actions, or services.
|
|
- Do not broaden workflow permissions, execute untrusted pull-request code with
|
|
credentials, or replace immutable action/tool pins with mutable tags.
|
|
- Changes to actions or standalone tools must update [`ci/digests.txt`](ci/digests.txt).
|
|
- Changes to dependencies must include the reviewed `uv.lock` and relevant license,
|
|
vulnerability, and SBOM effects.
|
|
|
|
Report vulnerabilities privately as described in [`SECURITY.md`](SECURITY.md).
|
|
Do not open a public issue for an undisclosed vulnerability.
|
|
|
|
## Research integrity and experiment evidence
|
|
|
|
Research, evaluation, and performance claims must be reproducible and scoped to the
|
|
evidence actually collected. Include, as applicable:
|
|
|
|
- exact commit, model and dataset identifiers, immutable revisions, dependency lock,
|
|
configuration, seed, hardware, and commands;
|
|
- raw or retained machine-readable output and hashes, not only a prose summary;
|
|
- baseline and comparison method, metric definition, uncertainty, failure cases, and
|
|
known limitations;
|
|
- citations to primary sources and provenance for imported claims or artifacts;
|
|
- a clear label for contributor-reported results that maintainers could not
|
|
independently reproduce.
|
|
|
|
Do not generalize a result beyond tested models, hardware, operating systems, or
|
|
backends. Performance claims need measured evidence; availability or a successful
|
|
import is not a performance result.
|
|
|
|
## Pull-request requirements
|
|
|
|
A merge-ready pull request:
|
|
|
|
- solves one coherent problem and links its canonical issue where one exists;
|
|
- explains user-visible behavior, risk surfaces, trust-boundary changes, and
|
|
compatibility impact;
|
|
- lists exact commands and outcomes, including checks not run and why;
|
|
- includes relevant code, tests, documentation, policy/risk-map updates, and lock or
|
|
digest changes in the same reviewable unit;
|
|
- contains no generated provider files, caches, unrelated cleanup, or drive-by
|
|
refactors;
|
|
- has signed commits, a current exact head, no unresolved review threads, no merge
|
|
conflicts, and every required CI check green.
|
|
|
|
Review is performed at an immutable head SHA. Pushing new commits invalidates prior
|
|
test and review conclusions until the new head is checked. Maintainers may split or
|
|
re-derive a stale legacy pull request and preserve its authorship, but new pull
|
|
requests are expected to arrive with their complete relevant test suite.
|
|
|
|
## Contributing experiment results
|
|
|
|
To contribute an abliteration result to the community dataset:
|
|
|
|
```bash
|
|
obliteratus obliterate <model> --method advanced --contribute \
|
|
--contribute-notes "Hardware: A100, prompt set: default"
|
|
```
|
|
|
|
Review the generated JSON for accidental sensitive data, add only the intended
|
|
`community_results/*.json` files, and include the provenance fields described above.
|
|
Preview aggregate output with:
|
|
|
|
```bash
|
|
obliteratus aggregate --format summary
|
|
obliteratus aggregate --format latex --min-runs 3
|
|
```
|
|
|
|
## Reporting bugs
|
|
|
|
Open an issue with expected and actual behavior, minimal reproduction, OBLITERATUS
|
|
version or commit, operating system, Python and dependency versions, model and
|
|
immutable revision where relevant, hardware/backend, and sanitized logs. Never post
|
|
credentials, private model data, or undisclosed security details.
|
|
|
|
## License
|
|
|
|
By contributing, you agree that your contributions are licensed under
|
|
[`AGPL-3.0-or-later`](LICENSE).
|