CalvinBackup/OBLITERATUS
1
0
Fork 0
mirror of https://github.com/elder-plinius/OBLITERATUS.git synced 2026-08-17 16:37:30 +02:00
Code Issues Packages Projects Releases Wiki Activity

docs: publish Gate 3 final baseline

Browse Source
This commit is contained in:
Joseph Magly
2026-08-16 10:48:21 -04:00
parent 42b30f7e5b
commit d4e029eb29
3 changed files with 163 additions and 33 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.aiwg/testing/gate3-final-report.md
+110
View File
@@ -0,0 +1,110 @@
# Gate 3 final baseline and release report
Date: 2026-08-16
Exact measured canonical commit:
`42b30f7e5b8ee596b3b0b039b10af16f01deec1e`
Integration: PR #111, signature-preserving fast-forward
Status: measured criteria PASS; final report PR audit and canonical
reconciliation remain required before ordinary feature work resumes
## Exit-criterion results
| Gate | Canonical result |
|---|---|
| Mandatory Python 3.12 lane | 2,119 selected tests passed, 9 conditional tests deselected by policy, and 0 failed, errored, or skipped in 123.069s |
| Repository coverage | 83.49% statements / 71.04% branches; floors are 80% / 65% |
| Mature CPU coverage | 94.02% statements / 84.54% branches; immutable floors are 94% / 84% |
| Changed executable lines | 95% blocking floor; Gate 3 production increments met the floor, including 58/58 on item 7 |
| Selected mutation | 1,844/2,057 killed (89.65%); 208 survived and 5 timed out; blocking floor is 85% |
| Repeatability | 702 tests passed in each of three order/hash-seed runs; 0 flakes, failures, or skips; 50.907s total |
| Duration | Complete Python 3.12 suite 123.069s, repeat gate 50.907s, mutation 1,872.35s; all within their 10m/5m/45m budgets |
| Portability and delivery | Python 3.10/3.11/3.12, Windows checkpoint contracts, package, Ruff/actionlint, and supply chain all passed |
| Software conditional evidence | Exact-head run 31950226930 and canonical run 31952128889 passed policy, pinned model, external evaluation, loopback network, operator UI, and final freshness summary |
| Accelerator operator evidence | Titan RTX 4090 passed CUDA and bitsandbytes; Mutsu Apple M4 / 16 GB passed MPS and MLX at signed implementation head `fa233fd8c97d2a1463141535f2a288b264abe93b` |
| Unscheduled environments | GitHub accelerator-runner registration and remote execution remain explicit no-support/no-correctness/no-performance waivers through 2026-09-15 under issue #110 |
| Review integrity | PR #111 was audited at immutable head, had no findings or review threads, passed all eight mandatory jobs, and preserved four publish-key signatures on `main` |
The canonical mandatory run for the measured commit is
https://github.com/elder-plinius/OBLITERATUS/actions/runs/31952112615.
It passed all eight mandatory jobs on the first attempt. The result above also
matches the successful exact-merge candidate run
https://github.com/elder-plinius/OBLITERATUS/actions/runs/31950217640.
## Delivered increments
| Item | Canonical commit | Delivery and protected behavior |
|---:|---|---|
| 1 | `256c39ea6a492749a4db44146830e9d78fd3fed8` | Adopted quantitative Gate 3 policy, duration budgets, and initial measurements |
| 2 | `2c2b38b501f94af46ffac3a772f7bf9475154dc5` | Added independent numerical, property, and metamorphic oracles (PR #102) |
| 3 | `c1b34503ddd3cb797e5d70671c47afcabfe73832` | Added loader, architecture, dtype, quantization, and shared-weight decision contracts (PR #103) |
| 4 | `d7d2ad566af778b1d315616bbea8b8a3c4dd2191` | Added checkpoint failure injection, atomicity, retry, cleanup, concurrency, and Windows contracts (PR #104) |
| 5 | `aa182cc44883890c019c393f17602e2e2702e7d7` | Added BESTIARY, model-client, transport, and watchtower state contracts (PR #105) |
| 6 | `9683e0be4d892e6a6a134b45a065e591e139da3c` | Added tournament, interactive, and UI decision-seam contracts (PR #108) |
| 7 | `b961623513a0e137b959f608ce31511d59e85888` | Added the tiny-model vertical slice and safe quantized/tied-weight restoration semantics (PR #109) |
| 8 | `42b30f7e5b8ee596b3b0b039b10af16f01deec1e` | Added conditional evidence governance, expiring claim waivers, and a functional same-version CUDA Torch overlay (PR #111) |
The AIWG workspace and CI contract were also refreshed and reconciled at
`e8ac3b65670d696ed5b68f06adf14706bc2ff865` (PR #107). The deployed AIWG
version is 2026.8.11, and the generated provider context continues to route
tracker and delivery authority through `.aiwg/aiwg.config`.
## Evidence integrity and claim boundary
The mandatory CI run tests the GitHub pull-request merge candidate; separate
manual conditional runs bind their JSON and JUnit evidence to the exact source
commit. Every selected software conditional record for item 8 names
`42b30f7e5b8ee596b3b0b039b10af16f01deec1e`, reports zero failures, errors,
or skips, and the summary has an empty failure list.
The Titan and Mutsu probes used isolated temporary directories,
checksum-verified uv 0.12.4 binaries, managed Python 3.12.13, the exact signed
source archive, and the committed dependency lock. Their temporary remote
directories were removed after evidence retrieval. These probes establish only
the recorded operations. They do not replace scheduled GitHub lanes or create
a broad accelerator support, compatibility, correctness, or performance claim.
Issue https://github.com/elder-plinius/OBLITERATUS/issues/110 remains open for
durable GitHub runner registration and least-privileged remote-execution
coverage. Waivers fail closed if malformed, expired, longer than 30 days, or
detached from that canonical tracker.
## Evidence locations
- PR #111 audit:
https://github.com/elder-plinius/OBLITERATUS/pull/111#issuecomment-5307860827
- Exact-head conditional run:
https://github.com/elder-plinius/OBLITERATUS/actions/runs/31950226930
- Canonical conditional run:
https://github.com/elder-plinius/OBLITERATUS/actions/runs/31952128889
- Downloaded exact-head conditional artifacts:
`/tmp/obliteratus-item8-final-head-eJ79m7`
- Downloaded exact-head quality artifacts:
`/tmp/obliteratus-item8-quality-au2hsV`
- Downloaded canonical CI and conditional artifacts:
`/tmp/obliteratus-item8-canonical-b7MJu4`
- Exact-head hardware JSON/JUnit evidence:
`/tmp/obliteratus-item8-hardware-evidence-GqqBSm`
- Hardware evidence and waiver tracker comment:
https://github.com/elder-plinius/OBLITERATUS/issues/110#issuecomment-5307675226
- Per-increment detail:
`gate3-increment-2-report.md` through `gate3-increment-8-report.md`
Canonical SHA-256 anchors are `143c42b579c91f5dc6d1d82fd74d10c0295056dd913986da178eb22b4e491275`
for mutation statistics, `11038dbd3f27c152cf057a3ae806a845cefab8dec9b55fbd26b5fe94c4cf21bb`
for the quality trend, `3b69fe70763b0c8fc82875a4bc9febca203a7b4c384be4605c7d1746ae0ab230`
for Python 3.12 coverage, and
`bc72408d88b4d2c480ebfaeae958a4ad0db0d13790adaa1ac52aee1f92e48671`
for the conditional summary.
## Release decision
The measured canonical commit satisfies every quantitative Gate 3 CPU,
coverage, mutation, repeatability, portability, delivery, and fresh-software
conditional criterion. There are no active flaky quarantines and no selected
gate was converted to green through a skip or retry.
The remaining accelerator and remote gaps are visible, owned, expiring, and
claim-blocking rather than silently treated as support evidence. Subject to a
clean exact-head audit, mandatory CI, signed integration, and post-merge
reconciliation for this report-only item 9, the testing-depth pause may end and
the contributor merge train may resume one immutable PR head at a time.
.aiwg/testing/gate3-increment-8-report.md
+22 -8
View File
@@ -3,8 +3,13 @@
Date: 2026-08-16
Exact base: `b961623513a0e137b959f608ce31511d59e85888`
Exact policy implementation head: `fa233fd8c97d2a1463141535f2a288b264abe93b`
Status: PASS locally, on exact-head software infrastructure, and on Titan/Mutsu
operator hardware; hosted PR verification and canonical merge remain pending
Exact canonical head: `42b30f7e5b8ee596b3b0b039b10af16f01deec1e`
Canonical mandatory run:
https://github.com/elder-plinius/OBLITERATUS/actions/runs/31952112615
Canonical conditional run:
https://github.com/elder-plinius/OBLITERATUS/actions/runs/31952128889
Status: PASS; PR #111 was exact-head audited, signature-preserving merged, and
canonically reconciled
## Results
@@ -14,13 +19,14 @@ operator hardware; hosted PR verification and canonical merge remain pending
| Policy regression set | 110 policy and CI-contract tests passed |
| Mandatory CPU lane | Python 3.12: 2,122 passed and 6 environment-dependent tests skipped in 81.44s; repository coverage 83.51% statements |
| Policy validation | Committed conditional policy passes direct CLI validation on 2026-08-16 |
| Software conditional gates | Exact-head run 31949881557 passed pinned-model runtime, external evaluation, loopback network, operator UI, policy, and final summary |
| Software conditional gates | Exact final-head run 31950226930 and canonical run 31952128889 passed pinned-model runtime, external evaluation, loopback network, operator UI, policy, and final summary |
| Canonical mandatory gates | Run 31952112615 passed Python 3.10/3.11/3.12, Windows, package, Ruff/actionlint, supply chain, repeatability, and mutation depth |
| CUDA and bitsandbytes | Titan RTX 4090: Torch 2.13.0+cu130 / CUDA 13.0; both gate invocations passed 2/2 at exact head; GitHub runner registration remains waived through 2026-09-15 under issue #110 |
| Apple MPS | Mutsu Apple M4 / 16 GB: native Torch 2.13.0 reports MPS built and available; gate passed 1/1 at exact head; GitHub runner registration remains waived through 2026-09-15 |
| Apple MLX | Mutsu Apple M4 / 16 GB: MLX 0.32.0 and MLX-LM 0.31.3; gate passed 1/1 at exact head; GitHub runner registration remains waived through 2026-09-15 |
| Remote execution | Required repository variables and secrets are unset; waived through 2026-09-15 under issue #110; no support, compatibility, correctness, or performance claim |
| Runner inventory | Titan exposes Gitea GPU/host runners only; Mutsu exposes a Colima builder but no runner process; the GitHub token also receives HTTP 403 from the repository-runner inventory API |
| Signing | Policy commit is signed and verified with publish-key fingerprint `62297562B1C7053088F405DB0117DAAA677A5BF2` |
| Signing | All four item-8 commits are signed and verified with publish-key fingerprint `62297562B1C7053088F405DB0117DAAA677A5BF2` |
## Contracts added
@@ -74,9 +80,15 @@ register maintainable GitHub runners.
## Verification and evidence
- Exact-head software conditional run:
https://github.com/elder-plinius/OBLITERATUS/actions/runs/31949881557
https://github.com/elder-plinius/OBLITERATUS/actions/runs/31950226930
- Canonical software conditional run:
https://github.com/elder-plinius/OBLITERATUS/actions/runs/31952128889
- Canonical mandatory run:
https://github.com/elder-plinius/OBLITERATUS/actions/runs/31952112615
- Downloaded software JSON/JUnit evidence:
`/tmp/obliteratus-item8-software-final-GMGS0x`
`/tmp/obliteratus-item8-final-head-eJ79m7`
- Downloaded canonical CI and conditional evidence:
`/tmp/obliteratus-item8-canonical-b7MJu4`
- Exact-head hardware JSON/JUnit and SHA-256 evidence:
`/tmp/obliteratus-item8-hardware-evidence-GqqBSm`
- Focused local command:
@@ -89,11 +101,13 @@ register maintainable GitHub runners.
https://github.com/elder-plinius/OBLITERATUS/issues/110
- Hardware-evidence tracker comment:
https://github.com/elder-plinius/OBLITERATUS/issues/110#issuecomment-5307675226
- Exact-head audit:
https://github.com/elder-plinius/OBLITERATUS/pull/111#issuecomment-5307860827
## Scope and release decision
This increment changes conditional-test governance and CUDA environment setup.
It does not alter production behavior or weaken mandatory CPU, package,
coverage, repeat, mutation, Windows, or supply-chain gates. Gate 3 item 9 and
the final exact-canonical report remain required before the contributor merge
train resumes.
its final report audit and canonical reconciliation remain required before the
contributor merge train resumes.
.aiwg/testing/master-test-plan.md
+31 -25
View File
@@ -27,26 +27,30 @@ Gate 3 increment 6 canonical commit:
Gate 3 increment 7 canonical commit:
`b961623513a0e137b959f608ce31511d59e85888` (PR #109, signed fast-forward)
Gate 3 increment 8 canonical commit:
`42b30f7e5b8ee596b3b0b039b10af16f01deec1e` (PR #111, signed fast-forward)
Current canonical AIWG and CI-contract baseline:
`e8ac3b65670d696ed5b68f06adf14706bc2ff865` (PR #107, rebase merged)
The latest canonical mandatory offline CPU lane passes 2,112 tests on Python
The latest canonical mandatory offline CPU lane passes 2,119 tests on Python
3.12 with 9 conditional tests deselected by policy, no failures, errors, skips,
or unexpected warnings, and 83.50% statement / 71.08% branch coverage. Mature
or unexpected warnings, and 83.49% statement / 71.04% branch coverage. Mature
CPU-testable coverage is 94.02% / 84.54%. The canonical mutation campaign kills
1,844 of 2,057 mutants (89.65%), with 208 survivors and 5 timeouts, while 702
tests pass in each of three file orders and hash seeds with no flakes. Package,
Windows checkpoint, supply-chain, and all other hosted jobs passed on attempt 1:
https://github.com/elder-plinius/OBLITERATUS/actions/runs/31947837466.
https://github.com/elder-plinius/OBLITERATUS/actions/runs/31952112615.
Gate 3 increment 8 is implemented at signed policy head
`fa233fd8c97d2a1463141535f2a288b264abe93b`. Fresh pinned-model download,
external-evaluation, loopback-network, and operator-UI evidence passes at that
exact head in run 31949881557. Titan CUDA/bitsandbytes and Mutsu MPS/MLX operator
probes also pass at that exact head. GitHub runner registration and remote
execution remain time-bounded no-support/no-correctness/no-performance waivers
through 2026-09-15 under issue #110. Details are in
`gate3-increment-8-report.md`.
Gate 3 increment 8 is canonical at signed head
`42b30f7e5b8ee596b3b0b039b10af16f01deec1e`. Fresh pinned-model download,
external-evaluation, loopback-network, operator-UI, and final-summary evidence
passes at that exact head in runs 31950226930 and 31952128889. Titan
CUDA/bitsandbytes and Mutsu MPS/MLX operator probes pass at signed implementation
head `fa233fd8c97d2a1463141535f2a288b264abe93b`. GitHub runner registration and
remote execution remain time-bounded no-support/no-correctness/no-performance
waivers through 2026-09-15 under issue #110. Details are in
`gate3-increment-8-report.md` and `gate3-final-report.md`.
The foundation, Wave A, Gate 1, and Gate 2 establish a sane,
evidence-producing baseline. The legacy contributor merge train proceeded one
@@ -55,8 +59,9 @@ property, mutation, determinism, and real-environment evidence without weakening
the mandatory CPU baseline.
The operator has now placed ordinary feature work back behind a testing-depth
gate. Gate 3 is adopted and items 1–7 are canonical; conditional reconciliation
and the final canonical report must pass before unrelated feature work resumes.
gate. Gate 3 is adopted and items 1–8 are canonical; the final report must pass
exact-head review and canonical reconciliation before unrelated feature work
resumes.
Correctness, security, data-loss, and test-infrastructure repairs remain
permitted when narrowly scoped and paired with reproducing tests.
@@ -295,15 +300,15 @@ resistance, environment evidence, or deterministic replay.
| Measure | Current | Gate 3 minimum |
|---|---:|---:|
| Repository statements | 83.50% item-7 candidate | 80.00% |
| Repository branches | 71.08% item-7 candidate | 65.00% |
| Mature CPU statements | 94.02% item-7 candidate | 94.00% |
| Mature CPU branches | 84.54% item-7 candidate | 84.00% |
| Repository statements | 83.49% item-8 canonical | 80.00% |
| Repository branches | 71.04% item-8 canonical | 65.00% |
| Mature CPU statements | 94.02% item-8 canonical | 94.00% |
| Mature CPU branches | 84.54% item-8 canonical | 84.00% |
| Changed executable lines | 95.00% floor | 95.00% floor |
| Selected mutation score | 89.65% item-7 candidate; 1,844/2,057 killed | at least 85% on expanded scope |
| Selected mutation score | 89.65% item-8 canonical; 1,844/2,057 killed | at least 85% on expanded scope |
| Unexpected warnings | 0 | 0 |
| Active flaky quarantines | 0 | 0 |
| Software conditional evidence age | 1 day at the item-6 candidate date | at most 8 days |
| Software conditional evidence age | same-day item-8 canonical evidence | at most 8 days |
The expanded mutation scope must include core mutation math, architecture and
loader decisions, persistence/atomicity, evaluation/report serialization, and
@@ -400,12 +405,13 @@ single repository-wide coverage PR.
Float-to-integer restoration regression. The canonical head reaches 94.02%
mature CPU statements and 84.54% branches, raises the immutable floors to
94% / 84%, and passes all eight hosted checks.
8. **Implemented at signed policy head `fa233fd`.** Reconcile fresh software
evidence and explicitly waive each unavailable hardware/remote environment
under the claim policy. Exact-head conditional evidence, PR audit, and
canonical merge remain pending.
9. Publish the Gate 3 execution, coverage, mutation, repeat, duration, and
conditional-evidence reports at one canonical `main` commit.
8. **Completed at `42b30f7` (PR #111).** Reconcile fresh software evidence,
run exact-head Titan/Mutsu operator probes, and explicitly waive unavailable
scheduled runner/remote lanes under the claim policy. The signed commits
were preserved by fast-forward integration.
9. **In progress.** Publish the Gate 3 execution, coverage, mutation, repeat,
duration, and conditional-evidence report, then exact-head audit, signed
merge, and canonically reconcile the report-only commit.
#### Per-PR acceptance evidence