docs: record Gate 3 increment 7 evidence

This commit is contained in:
Joseph Magly
2026-08-16 07:53:41 -04:00
parent acc6b3b254
commit 04e757eda8
3 changed files with 150 additions and 21 deletions
+15 -11
View File
@@ -4,8 +4,9 @@ Date: 2026-08-16
Exact base: `e8ac3b65670d696ed5b68f06adf14706bc2ff865`
Exact core implementation and local evidence head:
`77adf119f871ba820f1f02d55fe2760aa24c84b2`
Status: PASS locally; exact-head PR audit, hosted matrix, and canonical merge
remain pending
Exact canonical head: `9683e0be4d892e6a6a134b45a065e591e139da3c`
Canonical run: https://github.com/elder-plinius/OBLITERATUS/actions/runs/31941334488
Status: PASS; PR #108 was exact-head audited and canonically merged
## Results
@@ -26,9 +27,12 @@ remain pending
| Repeatability | 669 tests passed in each of three file orders/hash seeds; 35.353s total; no flake candidates or consistent failures |
| Policy | Ruff F, conditional policy, source-to-test risk map, mutation-target guard, global/mature coverage, changed-line, and touched-module gates passed |
The repository statement and branch exits now both exceed the Gate 3 targets.
The remaining new-work hold is bounded item 7, followed by conditional-evidence
reconciliation and final canonical reporting.
The repository statement and branch exits both exceed the Gate 3 targets. The
canonical run passed all eight required checks, including 2,059 tests on each
of Python 3.10, 3.11, and 3.12; 89.50% selected mutation score (1,816/2,029);
three repeat orders with 669 tests each and no flakes; Windows checkpoint,
package, and supply-chain evidence. The remaining new-work hold is bounded item
7, followed by conditional-evidence reconciliation and final reporting.
## Contracts added
@@ -101,12 +105,12 @@ root/model-directory resolution, raising the final changed-line result to
- Mutation metadata and prepared selection evidence:
ignored worktree-local `mutants/` artifacts at the exact implementation head
The exact implementation head passes the enforced Ruff F scope, immutable
quality policy, conditional-policy validation, risk-map validation, mutation
required-target guard, 75% statement and 60% branch floors, 95% changed-line
floor, and touched-module no-regression gate. The full 2,029-mutant campaign,
package matrix, supply-chain checks, exact-head audit, and canonical evidence
remain hosted PR/merge gates.
The exact implementation and canonical heads pass the enforced Ruff F scope,
immutable quality policy, conditional-policy validation, risk-map validation,
mutation required-target guard, 75% statement and 60% branch floors, 95%
changed-line floor, and touched-module no-regression gate. The hosted
2,029-mutant campaign, package matrix, supply-chain checks, exact-head audit,
and canonical reconciliation are complete.
## Scope and release decision
+120
View File
@@ -0,0 +1,120 @@
# Gate 3 increment 7 tiny-runtime and quantized-semantics report
Date: 2026-08-16
Exact base: `9683e0be4d892e6a6a134b45a065e591e139da3c`
Exact implementation head: `acc6b3b254e6d358ddabe495b1ca79295dd36d70`
Status: PASS locally; signed exact-head PR audit, hosted matrix, and canonical
merge remain pending
## Results
| Gate | Result |
|---|---|
| Focused contracts | 267 restoration, runtime, offline-model, and policy tests passed; the final runtime set is 111/111 green |
| Mandatory CPU lane | Python 3.12: 2,112 passed, 9 conditional tests deselected, 0 failed, and 0 skipped in 87.70s |
| Repository coverage | 82.76% statements / 69.55% branches before; 83.50% / 71.08% after |
| Mature CPU scope | 93.27% / 83.06% before; 94.02% statements / 84.54% branches after |
| Immutable mature floor | Raised from 92% / 80% to 94% / 84% in policy and validator |
| Changed executable lines | 58/58 (100%) against the exact base before documentation-only changes |
| Core pipeline | `abliterate.py`: 50.91% / 38.03% before; 52.63% statements / 40.73% branches after |
| Adaptive defaults | 93.59% / 81.65% before; 100% statements / 89.87% branches after |
| Advanced metrics | 91.73% / 82.19% before; 100% statements / 98.61% branches after |
| Loader decisions | 92.14% / 90.00% before; 100% statements / 96.67% branches after |
| Runtime contracts | 100% statements / 100% branches, including the new restoration policy |
| Selected mutation | 1,844/2,057 killed (89.65%); 208 survived and 5 timed out outside changed logic; every restoration-helper mutant killed |
| Repeatability | 702 tests passed in each of three file orders/hash seeds in 36.857s; no flakes, skips, or consistent failures |
| Package | Wheel and sdist built at version 0.1.2; both pass Twine metadata validation |
| Policy | Ruff, immutable quality policy, conditional policy, risk map, mutation-target guard, global/mature coverage, and changed-line gates passed |
The item-7 candidate reaches every quantitative Gate 3 repository and mature
CPU exit criterion. Ordinary feature work remains held until this increment is
merged and the conditional-evidence and final-reporting items are reconciled.
## Contracts added
1. Multi-direction norm restoration measures logical dequantized weights,
restores once after the full subspace projection, caps amplification, and
fails closed for non-finite or degenerate norms.
2. Integer storage is promoted to a logical float parameter instead of casting
a projected float back to Byte/uint8 and erasing the result. The exact
Float-to-integer failure reported in issue #11 is covered.
3. Float and integer restoration preserve `Parameter` identity. Quantized
replacement rebinds every module alias to the same replacement parameter,
preserving tied/shared-weight semantics.
4. Packed modules use their repacker when available, explicitly materialize a
float weight with a warning when safe, and fail closed when they expose
neither a repacker nor writable logical weight.
5. A deterministic, generated GPT-2 fixture executes the real advanced
two-direction pipeline, saves and reloads the checkpoint, proves weights
changed, and independently verifies every transformer-layer matrix norm.
6. Adaptive-default knowledge lookup and merge boundaries, advanced-metric
tensor/refusal/KL/CKA boundaries, and loader import-compatibility failures
now have executable negative contracts rather than uncovered decision paths.
7. The source-to-test risk map now makes projection-math contracts mandatory
whenever the core abliteration pipeline changes.
## Defects detected and corrected
The pre-change norm restoration iterated raw parameters, converted integer
weights to float for arithmetic, and copied the result back to the original
integer dtype. That reproduces the reported PyTorch Float-to-Byte failure or,
where a cast succeeds, silently destroys fractional projection and rescaling
values. It also measured packed bytes instead of logical dequantized weights.
The corrected path classifies storage before restoration, uses the model's
dequantization and replacement seams, and keeps integer values in float storage
when no quantization scale/zero-point contract exists. The first tied-weight
review then exposed a second defect: replacing one quantized parameter could
break aliases. Alias rebinding now preserves the tie after replacement.
The changed-line gate initially reported 43/47 covered lines. Packed fallback,
degenerate no-op, and in-place float restoration contracts raised the final
production result to 58/58. Coverage review also found a duplicate unreachable
empty-sequence check in first-token KL validation; the shared tensor validator
already rejects that input, so the dead branch was removed.
## Verification and evidence
- Mandatory Python 3.12 coverage:
`/tmp/obliteratus-item7-final2-coverage-py3.12.json`
- Mandatory Python 3.12 JUnit:
`/tmp/obliteratus-item7-final2-junit-py3.12.xml`
- Exact-base Python 3.12 coverage:
`/tmp/obliteratus-main-9683e0b-zvr8hF/test-evidence-py3.12/coverage-py3.12.json`
- Clean non-editable Python 3.12 test environment:
`/tmp/obliteratus-item7-py312-env-AxUcVD`
- Mutation artifacts:
ignored worktree-local `mutants/` evidence prepared from the exact candidate;
final score 89.65% (1,844/2,057) with no changed-helper survivor
- Repeat evidence: `/tmp/obliteratus-item7-repeat-gate.json`
- Package artifacts: `/tmp/obliteratus-item7-package-QBOhVU/`; SHA-256
`34dcd464df0661bf63da3a9c4e406b7acbfa73e6fd09127aef6211d7a30a249e`
(wheel) and
`f487391c98e4931896238b38b5232fb2f7c08147ec90767328734bf04c5c5c69`
(sdist)
The first isolated sync selected unsupported Python 3.13 and was not used as
evidence. A separate reused Python 3.12 environment correctly failed two
installed-artifact provenance assertions because it imported a different
worktree. The final environment was recreated non-editably from this worktree,
its isolated import resolved to its own `site-packages`, and all 2,112 selected
tests passed.
## Signing and integration policy
Every candidate commit is signed with the configured repository publish key
fingerprint `62297562B1C7053088F405DB0117DAAA677A5BF2` through the approved host
GPG wrapper and verified locally with `git verify-commit`. Integration must
preserve that signature by exact fast-forward or create a locally signed merge
commit. GitHub rebase merge is not acceptable because it recreates unsigned
canonical commits.
## Scope and release decision
This increment uses a synthetic random-initialized offline model and isolated
temporary filesystems. It makes no network, external-model, accelerator,
credential, or publication claim. The exact-head PR must still pass all hosted
CPU, repeat, mutation, package, Windows, and supply-chain gates before a
signature-preserving merge. Conditional evidence reconciliation and final Gate
3 reporting remain open after merge, so ordinary feature work does not resume
yet.
+15 -10
View File
@@ -299,12 +299,12 @@ resistance, environment evidence, or deterministic replay.
| Measure | Current | Gate 3 minimum |
|---|---:|---:|
| Repository statements | 82.78% item-6 candidate | 80.00% |
| Repository branches | 69.62% item-6 candidate | 65.00% |
| Mature CPU statements | 93.27% item-6 candidate; 73 lines remain | 94.00% |
| Mature CPU branches | 83.06% item-6 candidate; 29 branches remain | 84.00% |
| Repository statements | 83.50% item-7 candidate | 80.00% |
| Repository branches | 71.08% item-7 candidate | 65.00% |
| Mature CPU statements | 94.02% item-7 candidate | 94.00% |
| Mature CPU branches | 84.54% item-7 candidate | 84.00% |
| Changed executable lines | 95.00% floor | 95.00% floor |
| Selected mutation score | 89.46% item-5 canonical; item-6 campaign expands to 2,029 mutants | at least 85% on expanded scope |
| Selected mutation score | 89.65% item-7 candidate; 1,844/2,057 killed | at least 85% on expanded scope |
| Unexpected warnings | 0 | 0 |
| Active flaky quarantines | 0 | 0 |
| Software conditional evidence age | 1 day at the item-6 candidate date | at most 8 days |
@@ -395,11 +395,16 @@ single repository-wide coverage PR.
5. **Completed at `aa182cc` (PR #105).** Add BESTIARY/model-client/watchtower
transport and scheduler state contracts. Exact evidence is recorded in
`gate3-increment-5-report.md` and the canonical PR follow-up.
6. **Implemented locally at `77adf11`.** Add tournament, interactive, and UI
decision-seam contracts. Exact local evidence is recorded in
`gate3-increment-6-report.md`; audit and canonical merge remain pending.
7. Extend the installed tiny-model vertical slice and quantized-storage
semantics, including the exact Float-to-integer restoration regression.
6. **Completed at `9683e0b` (PR #108).** Add tournament, interactive, and UI
decision-seam contracts. Canonical run 31941334488 passed all eight required
checks; exact evidence is recorded in `gate3-increment-6-report.md` and the
canonical PR reconciliation comment.
7. **Implemented locally at `acc6b3b` against `9683e0b`.** Extend the installed
tiny-model vertical slice and quantized-storage semantics, including the
exact Float-to-integer restoration regression. The candidate reaches 94.02%
mature CPU statements and 84.54% branches and raises the immutable floors
to 94% / 84%; signed commit, exact-head audit, hosted checks, and canonical
merge remain pending.
8. Reconcile conditional software evidence, then run or explicitly waive each
unavailable hardware/remote environment under the claim policy.
9. Publish the Gate 3 execution, coverage, mutation, repeat, duration, and