mirror of
https://github.com/elder-plinius/OBLITERATUS.git
synced 2026-08-30 14:40:38 +02:00
ci: adopt tiered contributor validation
This commit is contained in:
+55
-30
@@ -36,8 +36,9 @@ must be deliberate and follow [`docs/SUPPLY_CHAIN_POLICY.md`](docs/SUPPLY_CHAIN_
|
||||
|
||||
1. Start a focused branch from the current `main`.
|
||||
2. Make the smallest coherent change that solves the linked problem.
|
||||
3. Add tests with the implementation. New behavior without relevant tests is not
|
||||
merge-ready.
|
||||
3. Add focused tests with the implementation when practical. New behavior with
|
||||
zero relevant coverage cannot pass the PR gate; maintainers may add remaining
|
||||
test depth as an internal pre-merge task.
|
||||
4. Update user, operator, policy, risk-map, and research documentation affected by
|
||||
the change.
|
||||
5. Run the applicable local checks below.
|
||||
@@ -74,25 +75,32 @@ Tests must not download models or datasets, contact services, require an acceler
|
||||
launch a UI, or use remote credentials unless they are explicitly assigned to a
|
||||
conditional marker and gate.
|
||||
|
||||
CI currently requires:
|
||||
CI keeps the contributor gate intentionally small enough to return useful review
|
||||
feedback quickly. It currently requires:
|
||||
|
||||
- the full test matrix on Python 3.10, 3.11, and 3.12 with warnings treated as
|
||||
errors;
|
||||
- at least 75% repository line coverage and 60% repository branch coverage;
|
||||
- at least 95% coverage of changed executable lines;
|
||||
- no independent line or branch regression in a touched production module versus
|
||||
the exact base commit;
|
||||
- at least 80% line and 75% branch coverage for a new production module;
|
||||
- at least 70% line coverage for each policy-designated critical boundary module;
|
||||
- at least 94% line and 84% branch coverage for the mature CPU-testable scope;
|
||||
- at least an 85% score in the bounded selective mutation campaign;
|
||||
- zero unexpected warnings and no unowned, expired, or policy-invalid quarantine;
|
||||
- Ruff F, actionlint, package, installed-wheel, installed-sdist, supply-chain, and
|
||||
policy checks.
|
||||
- one offline Python 3.12 core lane with warnings treated as errors;
|
||||
- the versioned smoke suite, every changed CPU test, and contract tests selected
|
||||
from the exact diff through `ci/test-risk-map.json`;
|
||||
- at least 50% coverage of changed executable lines;
|
||||
- Ruff F, actionlint, lock, policy, import, and CLI contract checks;
|
||||
- a source and wheel build when package inputs change.
|
||||
|
||||
These values are floors, not targets. A change should strengthen behavioral
|
||||
confidence rather than consume existing margin.
|
||||
|
||||
A green pull-request gate means the submission is ready for maintainer review. It
|
||||
does not certify a release and does not guarantee immediate merge. Maintainers own
|
||||
the additional risk-driven tests, cross-version checks, packaging depth, and
|
||||
hardening needed for the merge decision.
|
||||
|
||||
### Tagged-release certification
|
||||
|
||||
The `v*` tag workflow, or an explicit manual release-validation run, executes the
|
||||
full Python 3.10-3.12 matrix and retains the stricter repository, branch,
|
||||
critical-module, touched-module, mature-scope, repeat, mutation, packaging,
|
||||
Windows, and supply-chain gates. The full suite remains mandatory for a release;
|
||||
the faster pull-request lane is not a release exemption.
|
||||
|
||||
### Test design standard
|
||||
|
||||
Each behavior-changing pull request must include:
|
||||
@@ -121,20 +129,33 @@ uv run --extra dev python scripts/check_conditional_policy.py
|
||||
|
||||
## Local validation
|
||||
|
||||
Run focused tests while developing, then the complete baseline before requesting
|
||||
review:
|
||||
Run the focused pull-request baseline while developing. Replace `origin/main` with
|
||||
the exact base commit when needed:
|
||||
|
||||
```bash
|
||||
uv lock --check
|
||||
uv run --extra dev python -m ruff check --select F app.py obliteratus tests scripts
|
||||
uv run --extra dev python -m pytest
|
||||
uv run --extra dev python -m build --sdist --wheel
|
||||
mkdir -p test-results
|
||||
uv run --extra dev python scripts/select_pr_tests.py --base-ref origin/main \
|
||||
> test-results/selected-tests.txt
|
||||
mapfile -t selected_tests < test-results/selected-tests.txt
|
||||
uv run --extra dev python -m pytest "${selected_tests[@]}" \
|
||||
--cov=app --cov-branch --cov-fail-under=0 \
|
||||
--cov-report=json:test-results/coverage-pr-core.json
|
||||
uv run --extra dev python scripts/check_coverage_thresholds.py \
|
||||
test-results/coverage-pr-core.json --min-line 0 --min-branch 0 \
|
||||
--min-changed 50 --base-ref origin/main
|
||||
uv run --extra dev python -c 'import obliteratus; print(obliteratus.__version__)'
|
||||
uv run --extra dev python -m obliteratus --help
|
||||
uv run --extra dev python scripts/check_conditional_policy.py
|
||||
uv run --extra dev python scripts/check_test_risk_map.py
|
||||
```
|
||||
|
||||
If package inputs changed, also run
|
||||
`uv run --extra dev python -m build --sdist --wheel`. Maintainers run the complete
|
||||
`python -m pytest`, package-install,
|
||||
cross-version, and quality-depth suite before a tagged release.
|
||||
|
||||
Ruff E501 remains a non-blocking legacy-debt report; new code should still respect
|
||||
the configured 100-character line length. The exact reporting command lives in CI.
|
||||
|
||||
@@ -156,8 +177,9 @@ uv run --extra dev --group quality python scripts/check_mutation_score.py \
|
||||
mutants/mutmut-cicd-stats.json --minimum 85
|
||||
```
|
||||
|
||||
CI remains authoritative for exact-base coverage comparison, actionlint, platform
|
||||
matrix results, installed-distribution checks, and retained evidence.
|
||||
CI remains authoritative for exact-base changed-line coverage, actionlint, selected
|
||||
test evidence, release platform-matrix results, installed-distribution checks, and
|
||||
retained evidence.
|
||||
|
||||
## Risk-specific checks
|
||||
|
||||
@@ -234,23 +256,26 @@ import is not a performance result.
|
||||
|
||||
## Pull-request requirements
|
||||
|
||||
A merge-ready pull request:
|
||||
A review-ready pull request:
|
||||
|
||||
- solves one coherent problem and links its canonical issue where one exists;
|
||||
- explains user-visible behavior, risk surfaces, trust-boundary changes, and
|
||||
compatibility impact;
|
||||
- lists exact commands and outcomes, including checks not run and why;
|
||||
- includes relevant code, tests, documentation, policy/risk-map updates, and lock or
|
||||
digest changes in the same reviewable unit;
|
||||
- includes relevant code and whatever focused tests the contributor can supply,
|
||||
plus documentation, policy/risk-map updates, and lock or digest changes in the
|
||||
same reviewable unit;
|
||||
- contains no generated provider files, caches, unrelated cleanup, or drive-by
|
||||
refactors;
|
||||
- has signed commits, a current exact head, no unresolved review threads, no merge
|
||||
conflicts, and every required CI check green.
|
||||
- has signed commits, a current exact head, no merge conflicts, and every
|
||||
contributor-facing CI check green.
|
||||
|
||||
Review is performed at an immutable head SHA. Pushing new commits invalidates prior
|
||||
test and review conclusions until the new head is checked. Maintainers may split or
|
||||
re-derive a stale legacy pull request and preserve its authorship, but new pull
|
||||
requests are expected to arrive with their complete relevant test suite.
|
||||
test and review conclusions until the new head is checked. Maintainers may add
|
||||
tests or hardening in a separate attributable commit, resolve review threads, and
|
||||
run deeper gates before merge. Behavior changes with no relevant test coverage
|
||||
remain blocked; contributors are not expected to satisfy release-depth coverage
|
||||
and mutation gates merely to request review.
|
||||
|
||||
## Contributing experiment results
|
||||
|
||||
|
||||
Reference in New Issue
Block a user