mirror of
https://github.com/elder-plinius/OBLITERATUS.git
synced 2026-08-17 16:37:30 +02:00
ci: verify checkpoint portability on Windows
This commit is contained in:
@@ -5,8 +5,10 @@ Exact base: `2c2b38b501f94af46ffac3a772f7bf9475154dc5`
|
||||
Exact implementation/evidence head: `e2e120c211adf3ec9a7909a401f5e7a34f17984f`
|
||||
Delivery head: the containing documentation commit, resolved with
|
||||
`git rev-parse HEAD`
|
||||
Canonical merge: pending
|
||||
Status: PASS locally; hosted exact-head audit and merge pending
|
||||
Canonical merge: `c1b34503ddd3cb797e5d70671c47afcabfe73832` (PR #103,
|
||||
rebase merged)
|
||||
Status: PASS; exact-head audit, rebase merge, and canonical post-merge CI
|
||||
complete
|
||||
|
||||
## Results
|
||||
|
||||
@@ -88,6 +90,9 @@ decision logic and its wrappers only; CUDA, bitsandbytes hardware execution,
|
||||
MPS, MLX, network, download, remote, and operator-UI mappings are unchanged, so
|
||||
no new environment support claim is made.
|
||||
|
||||
Delivery requires the containing documentation commit to be pushed, audited at
|
||||
its final SHA, pass all hosted checks with no unresolved review thread, and be
|
||||
rebase merged before item 4 begins.
|
||||
PR #103 passed its exact-head audit and all required hosted checks with no
|
||||
unresolved review thread, then rebase merged at
|
||||
`c1b34503ddd3cb797e5d70671c47afcabfe73832`. All seven jobs in canonical
|
||||
post-merge run
|
||||
https://github.com/elder-plinius/OBLITERATUS/actions/runs/31920984688 passed
|
||||
before item 4 began.
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
# Gate 3 increment 4 checkpoint reliability report
|
||||
|
||||
Date: 2026-08-15
|
||||
Exact base: `c1b34503ddd3cb797e5d70671c47afcabfe73832`
|
||||
Exact core implementation/mutation head:
|
||||
`9da022e6f3bc73d17fd31d4da70b42e3ca052ba2`
|
||||
Delivery head: the containing documentation and Windows-CI commit, resolved with
|
||||
`git rev-parse HEAD`
|
||||
Canonical merge: pending
|
||||
Status: PASS locally; hosted Windows contract, exact-head audit, and merge pending
|
||||
|
||||
## Results
|
||||
|
||||
| Gate | Result |
|
||||
|---|---|
|
||||
| Focused checkpoint/policy contracts | 176 passed in 6.08s at the exact core head; the delivery-head portability-policy selection also passed 176 tests in 6.36s |
|
||||
| Mandatory CPU matrix | Exact core head — Python 3.10: 1,878 passed in 88.21s; Python 3.11: 1,878 passed in 80.63s. Exact delivery head — Python 3.12: 1,877 passed, 2 expected xfails in 45.49s. Nine conditional tests were deselected on each interpreter |
|
||||
| Repository coverage | 76.60% statements / 63.25% branches before; 76.77% / 63.55% after |
|
||||
| Mature CPU scope | 93.12% statements / 82.60% branches before; 93.19% / 82.85% after |
|
||||
| Changed executable lines | 138/138 (100%) against the exact base on Python 3.12 |
|
||||
| Touched production modules | 2/2 pass exact-base line and branch no-regression |
|
||||
| Persistence seam | `persistence_contracts.py`: 100% statements and 100% branches (166 statements, 58 branches) |
|
||||
| Abliteration runtime | 50.82% statements / 38.03% branches before; 50.91% / 38.03% after |
|
||||
| Selective mutation | 1,614/1,827 killed (88.34%); 208 survived; 5 timed out; 0 no-test, suspicious, skipped, interrupted, or segfault results |
|
||||
| Changed persistence mutation | 229/229 killed; 0 survived or timed out |
|
||||
| Mutation runtime | 920.27s; 1,121,616 KiB maximum parent RSS; required-target guard passed |
|
||||
| Repeatability | 493 tests passed in each of three file orders/hash seeds; 33.832s total; no flake candidates or consistent failures |
|
||||
| Package | sdist and wheel built; metadata, contents, isolated import, version, module entry point, and console entry point passed on Python 3.12 |
|
||||
| Supply chain | Pinned `uv 0.12.4` lock, all-extras import, Gitleaks, Python 3.10–3.12 vulnerability audits, wheel-bound CycloneDX SBOM, and license policy passed |
|
||||
| Windows portability | A focused Python 3.12 hosted job now blocks on checkpoint atomicity, persistence, and pipeline contracts and retains JUnit evidence; first exact-head result pending |
|
||||
|
||||
The mandatory Python lanes remain below the 240-second policy budget. The final
|
||||
Python 3.12 test time measured 10.08s faster than the exact-base 55.57s while
|
||||
adding 47 passing tests. The slowest Python 3.12 test remains the existing
|
||||
offline model save/reload slice at 5.364s; no new checkpoint test exceeds an
|
||||
owned duration budget.
|
||||
|
||||
## Contracts added
|
||||
|
||||
1. A staged checkpoint must contain the exact prepared metadata, non-empty JSON
|
||||
model and tokenizer configurations, and either regular direct weights or a
|
||||
complete safe sharded-weight map. Missing, empty, corrupt, truncated,
|
||||
link-backed, or traversal-bearing artifacts fail before promotion.
|
||||
2. Every regular staged artifact is flushed, nested directory entries are
|
||||
flushed bottom-up where the platform supports directory fsync, and the
|
||||
destination parent is flushed after backup, promotion, rollback, and durable
|
||||
cleanup transitions.
|
||||
3. A persistent sibling lock serializes cooperative threads and processes that
|
||||
commit to the same destination. Concurrent complete writers may replace one
|
||||
another, but readers never observe a partially promoted checkpoint.
|
||||
4. Writer failure and `BaseException` cancellation are injected before commit
|
||||
and at tree sync, backup replacement, backup-parent sync, promotion,
|
||||
promotion-parent sync, and rollback. Existing destinations are restored;
|
||||
newly created destinations are removed; ambiguous double failures fail
|
||||
closed with an explicit recovery path.
|
||||
5. Cleanup errors never mask the primary write failure. Only transaction-owned
|
||||
staging and backup paths are removed, and a successfully promoted checkpoint
|
||||
remains valid when obsolete-backup cleanup reports a durable warning.
|
||||
6. Pipeline offload cleanup no longer clears ownership after a failed removal.
|
||||
The owned path is retained for a deterministic retry, while successful
|
||||
cleanup clears both the path and ownership flag.
|
||||
7. The production rebirth path validates the complete staged checkpoint before
|
||||
atomic promotion. Persistence is now a required mutation target and the
|
||||
checkpoint atomicity/pipeline suites are part of mutation selection and the
|
||||
three-order repeat gate.
|
||||
8. The platform-specific lock and directory-sync branches are no longer backed
|
||||
by an unverified CI comment: a focused `windows-latest` lane executes the
|
||||
checkpoint contracts with the locked Python 3.12 environment and retained
|
||||
JUnit evidence.
|
||||
|
||||
## Defects and mutants detected
|
||||
|
||||
The development mutation campaign initially left 29 persistence survivors.
|
||||
They identified weak JSON type/path assertions, incomplete cleanup and
|
||||
diagnostic call contracts, missing cancellation boundaries, and insufficient
|
||||
parent-sync/rollback observability. The repaired exact-target campaign killed
|
||||
all generated persistence mutants. The final complete campaign preserved that
|
||||
229/229 result while exceeding the immutable aggregate 85% floor.
|
||||
|
||||
The full fault-injection suite also detects partial directory trees, unsafe or
|
||||
missing shards, symlinked artifacts, file-fsync failure, concurrent commit
|
||||
interleavings, incomplete rollback, cleanup-warning suppression, stale staging
|
||||
ownership, and offload-directory ownership loss.
|
||||
|
||||
## Verification and evidence
|
||||
|
||||
- Exact delivery-head Python 3.12 coverage/JUnit/trend evidence:
|
||||
`/tmp/obliteratus-gate3-item4-final-JlA1k3/`
|
||||
- Exact-base Python 3.12 coverage/JUnit evidence:
|
||||
`/tmp/obliteratus-gate3-item4-base-2uZEXr/`
|
||||
- Python 3.10 and 3.11 coverage/JUnit evidence:
|
||||
`/tmp/obliteratus-gate3-item4-py310-exact-ldLxx6/` and
|
||||
`/tmp/obliteratus-gate3-item4-py311-exact-H4wFBU/`
|
||||
- Repeat evidence:
|
||||
`/tmp/obliteratus-gate3-item4-repeat-H23MdN/`
|
||||
- Exact core-head mutation evidence:
|
||||
`/tmp/obliteratus-gate3-item4-mutation-PVKKOL/`
|
||||
- Python 3.10/3.11 and quality-depth normalized trends:
|
||||
`/tmp/obliteratus-gate3-item4-trends-lrrEjF/`
|
||||
- Package evidence:
|
||||
`/tmp/obliteratus-gate3-item4-package-CTZeC6/`
|
||||
- Pinned-tool supply-chain evidence:
|
||||
`/tmp/obliteratus-gate3-item4-supply-7aqGSc/`
|
||||
|
||||
The canonical Ruff F gate, checksum-verified `actionlint`, `uv lock --check`,
|
||||
conditional-policy validation, risk-map validation, changed-line gate,
|
||||
touched-module gate, immutable coverage policy, normalized duration policy,
|
||||
repeat gate, mutation score, installed-distribution checks, and supply-chain
|
||||
decisions all pass. The ordinary worktree remained clean; all work and evidence
|
||||
were isolated in the dedicated item-4 worktree and temporary evidence paths.
|
||||
|
||||
## Scope and release decision
|
||||
|
||||
This increment implements Gate 3 bounded delivery item 4. It does not reopen
|
||||
ordinary feature work: items 5–7, conditional-evidence reconciliation, and the
|
||||
final canonical Gate 3 publication remain open. The new hosted Windows lane is
|
||||
a checkpoint portability contract, not a claim that accelerator, MPS, MLX,
|
||||
network, download, remote, or operator-UI execution is supported by this
|
||||
increment.
|
||||
|
||||
Delivery requires the containing commit to be pushed, pass the first hosted
|
||||
Windows checkpoint run and all existing required checks, be audited at its
|
||||
exact final SHA with no unresolved review thread, and be rebase merged before
|
||||
item 5 begins.
|
||||
@@ -12,20 +12,30 @@ Gate 3 increment 1 canonical commit:
|
||||
Gate 3 increment 2 canonical commit:
|
||||
`2c2b38b501f94af46ffac3a772f7bf9475154dc5` (PR #102, rebase merged)
|
||||
|
||||
The mandatory offline CPU selection passes on Python 3.10–3.12 with 1,644
|
||||
tests, 9 conditionally deselected tests, no skips, and no warnings. The slowest
|
||||
hosted pytest selection is Python 3.12 at 158.989 seconds; its complete CI job
|
||||
is 5 minutes 41 seconds because it also regenerates exact-base coverage.
|
||||
Branch-enabled Python 3.12 coverage is 75.68% statements and 61.72% branches.
|
||||
The documented mature CPU-testable scope is 92.71% statements and 81.55%
|
||||
branches. The repository enforces 95% changed-line coverage, per-touched-module
|
||||
line and branch no-regression, a 75% selective mutation floor, zero unexpected
|
||||
warnings, and owned suite/test/marker/repeat duration budgets. The bounded
|
||||
campaign kills 809 of 886 mutants (91.31%), and 391 high-consequence tests pass
|
||||
in each of three file orders and hash seeds with no flake candidates. Package,
|
||||
installed wheel/sdist, vulnerability, secret, license, and SBOM checks are
|
||||
green. All seven jobs in the exact canonical-commit run passed:
|
||||
https://github.com/elder-plinius/OBLITERATUS/actions/runs/31874768085.
|
||||
Gate 3 increment 3 canonical commit:
|
||||
`c1b34503ddd3cb797e5d70671c47afcabfe73832` (PR #103, rebase merged)
|
||||
|
||||
The latest canonical mandatory offline CPU selection passes on Python 3.10–3.12
|
||||
with 1,830–1,832 passing tests, 9 conditionally deselected tests, 2 expected
|
||||
Python 3.12 xfails, no unexpected skips, and no warnings. Branch-enabled Python
|
||||
3.12 coverage is 76.60% statements and 63.25% branches; the documented mature
|
||||
CPU-testable scope is 93.12% statements and 82.60% branches. The repository
|
||||
enforces 95% changed-line coverage, per-touched-module line and branch
|
||||
no-regression, an 85% selective mutation floor, zero unexpected warnings, and
|
||||
owned suite/test/marker/repeat duration budgets. The canonical increment-3
|
||||
campaign killed 1,424 of 1,637 mutants (86.99%), and 431 high-consequence tests
|
||||
passed in each of three file orders and hash seeds. Package, installed
|
||||
wheel/sdist, vulnerability, secret, license, and SBOM checks are green. All
|
||||
seven jobs in the canonical post-merge run passed:
|
||||
https://github.com/elder-plinius/OBLITERATUS/actions/runs/31920984688.
|
||||
|
||||
Gate 3 increment 4 is implemented and locally verified at core evidence head
|
||||
`9da022e6f3bc73d17fd31d4da70b42e3ca052ba2`, with delivery evidence recorded in
|
||||
`gate3-increment-4-report.md`. Candidate coverage is 76.77% statements and
|
||||
63.55% branches; mature CPU coverage is 93.19% statements and 82.85% branches;
|
||||
the complete campaign kills 1,614 of 1,827 mutants (88.34%), including all
|
||||
229 persistence mutants. Exact-head PR audit, the first hosted Windows
|
||||
checkpoint contract, and canonical merge remain pending.
|
||||
|
||||
The latest conditional run is green for pinned-model download and evaluation,
|
||||
loopback network, operator UI, and evidence-freshness policy at the earlier
|
||||
@@ -42,8 +52,9 @@ property, mutation, determinism, and real-environment evidence without weakening
|
||||
the mandatory CPU baseline.
|
||||
|
||||
The operator has now placed ordinary feature work back behind a testing-depth
|
||||
gate. Gate 3 is adopted and items 1–2 are complete, but items 3–7 and the
|
||||
quantitative exit criteria must pass before unrelated feature work resumes.
|
||||
gate. Gate 3 is adopted and items 1–3 are complete; item 4 is implemented but
|
||||
not yet canonical; items 5–7 and the quantitative exit criteria must pass
|
||||
before unrelated feature work resumes.
|
||||
Correctness, security, data-loss, and test-infrastructure repairs remain
|
||||
permitted when narrowly scoped and paired with reproducing tests.
|
||||
|
||||
@@ -282,12 +293,12 @@ resistance, environment evidence, or deterministic replay.
|
||||
|
||||
| Measure | Current | Gate 3 minimum |
|
||||
|---|---:|---:|
|
||||
| Repository statements | 76.26% | 80.00% |
|
||||
| Repository branches | 62.67% | 65.00% |
|
||||
| Mature CPU statements | 92.87% | 94.00% |
|
||||
| Mature CPU branches | 82.24% | 84.00% |
|
||||
| Repository statements | 76.77% candidate | 80.00% |
|
||||
| Repository branches | 63.55% candidate | 65.00% |
|
||||
| Mature CPU statements | 93.19% candidate | 94.00% |
|
||||
| Mature CPU branches | 82.85% candidate | 84.00% |
|
||||
| Changed executable lines | 95.00% floor | 95.00% floor |
|
||||
| Selected mutation score | 86.15% on expanded numerical scope | at least 85% on expanded scope |
|
||||
| Selected mutation score | 88.34% candidate on expanded persistence scope | at least 85% on expanded scope |
|
||||
| Unexpected warnings | 0 | 0 |
|
||||
| Active flaky quarantines | 0 | 0 |
|
||||
| Software conditional evidence age | at most 8 days | at most 8 days |
|
||||
@@ -344,7 +355,7 @@ small pure seam and measure it rather than mocking away the behavior under test.
|
||||
|
||||
| Surface | Current statements / branches | Gate 3 intent |
|
||||
|---|---:|---|
|
||||
| `abliterate.py` | 49.63% / 37.31% | at least 52% / 45% with mutation/math contracts |
|
||||
| `abliterate.py` | 50.91% / 38.03% candidate | at least 52% / 45% with mutation/math contracts |
|
||||
| `lora_ablation.py` | 9.57% / 0% | at least 70% / 55% for CPU-testable decisions |
|
||||
| `tourney.py` | 12.01% / 0% | at least 55% / 40% for lifecycle/state transitions |
|
||||
| `bestiary_sync.py` | 0% / 0% | at least 80% / 65% using a loopback/fake transport |
|
||||
@@ -368,11 +379,13 @@ single repository-wide coverage PR.
|
||||
2. **Completed at `2c2b38b` (PR #102).** Add numerical/reference-oracle and
|
||||
metamorphic contracts; expand mutation to the corresponding pure math.
|
||||
Canonical evidence is recorded in `gate3-increment-2-report.md`.
|
||||
3. **Implemented in Gate 3 increment 3; exact-head PR audit and canonical merge
|
||||
pending.** Add loader, architecture, dtype, quantization, and shared-weight
|
||||
decision contracts with mutation evidence. Candidate evidence is recorded in
|
||||
`gate3-increment-3-report.md`.
|
||||
4. Add checkpoint failure injection, concurrency, atomicity, and retry contracts.
|
||||
3. **Completed at `c1b3450` (PR #103).** Add loader, architecture, dtype,
|
||||
quantization, and shared-weight decision contracts with mutation evidence.
|
||||
Canonical evidence is recorded in `gate3-increment-3-report.md`.
|
||||
4. **Implemented in Gate 3 increment 4; exact-head PR audit and canonical merge
|
||||
pending.** Add checkpoint failure injection, concurrency, atomicity, retry,
|
||||
cleanup-ownership, and Windows portability contracts. Candidate evidence is
|
||||
recorded in `gate3-increment-4-report.md`.
|
||||
5. Add BESTIARY/model-client/watchtower transport and scheduler state contracts.
|
||||
6. Add tournament, interactive, and UI decision-seam contracts.
|
||||
7. Extend the installed tiny-model vertical slice and quantized-storage
|
||||
|
||||
@@ -393,6 +393,61 @@ jobs:
|
||||
if-no-files-found: error
|
||||
retention-days: 90
|
||||
|
||||
checkpoint-windows:
|
||||
name: Checkpoint contracts (Windows)
|
||||
runs-on: windows-latest
|
||||
timeout-minutes: 15
|
||||
env:
|
||||
CUDA_VISIBLE_DEVICES: ""
|
||||
HF_DATASETS_OFFLINE: "1"
|
||||
HF_HUB_DISABLE_TELEMETRY: "1"
|
||||
HF_HUB_OFFLINE: "1"
|
||||
TOKENIZERS_PARALLELISM: "false"
|
||||
TRANSFORMERS_OFFLINE: "1"
|
||||
|
||||
steps:
|
||||
- name: Check out repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
cache: pip
|
||||
cache-dependency-path: |
|
||||
pyproject.toml
|
||||
uv.lock
|
||||
|
||||
- name: Install locked test environment
|
||||
shell: pwsh
|
||||
run: |
|
||||
python -m pip install "uv==$env:UV_VERSION"
|
||||
$testEnv = Join-Path $env:RUNNER_TEMP "obliteratus-checkpoint-windows"
|
||||
$env:UV_PROJECT_ENVIRONMENT = $testEnv
|
||||
uv sync --locked --no-default-groups --extra dev --no-editable
|
||||
"CHECKPOINT_TEST_ENV=$testEnv" >> $env:GITHUB_ENV
|
||||
|
||||
- name: Run Windows checkpoint contracts
|
||||
shell: pwsh
|
||||
run: |
|
||||
$python = Join-Path $env:CHECKPOINT_TEST_ENV "Scripts/python.exe"
|
||||
& $python -m pytest `
|
||||
tests/test_checkpoint_atomicity.py `
|
||||
tests/test_persistence_contracts.py `
|
||||
tests/test_persistence_pipeline.py `
|
||||
--no-cov `
|
||||
--junitxml="$env:RUNNER_TEMP/checkpoint-windows.xml" `
|
||||
-q
|
||||
|
||||
- name: Upload Windows checkpoint evidence
|
||||
if: always()
|
||||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
||||
with:
|
||||
name: checkpoint-evidence-windows-py3.12
|
||||
path: ${{ runner.temp }}/checkpoint-windows.xml
|
||||
if-no-files-found: error
|
||||
retention-days: 90
|
||||
|
||||
quality-depth:
|
||||
name: Quality depth
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
@@ -110,3 +110,21 @@ def test_ci_retains_normalized_test_and_quality_trends_for_ninety_days():
|
||||
assert "test-trend-py${{ matrix.python-version }}.json" in workflow
|
||||
assert "quality-trend-py3.12.json" in workflow
|
||||
assert workflow.count("retention-days: 90") >= 2
|
||||
|
||||
|
||||
def test_ci_runs_checkpoint_portability_contracts_on_windows():
|
||||
workflow = WORKFLOW.read_text(encoding="utf-8")
|
||||
checkpoint_job = workflow.split(" checkpoint-windows:\n", maxsplit=1)[1].split(
|
||||
" quality-depth:\n",
|
||||
maxsplit=1,
|
||||
)[0]
|
||||
|
||||
assert "runs-on: windows-latest" in checkpoint_job
|
||||
assert 'python-version: "3.12"' in checkpoint_job
|
||||
assert "uv sync --locked --no-default-groups --extra dev --no-editable" in checkpoint_job
|
||||
assert "tests/test_checkpoint_atomicity.py" in checkpoint_job
|
||||
assert "tests/test_persistence_contracts.py" in checkpoint_job
|
||||
assert "tests/test_persistence_pipeline.py" in checkpoint_job
|
||||
assert "--no-cov" in checkpoint_job
|
||||
assert "checkpoint-evidence-windows-py3.12" in checkpoint_job
|
||||
assert "if-no-files-found: error" in checkpoint_job
|
||||
|
||||
Reference in New Issue
Block a user