mirror of
https://github.com/elder-plinius/OBLITERATUS.git
synced 2026-08-17 16:37:30 +02:00
docs: record Gate 3 increment 7 evidence
This commit is contained in:
@@ -4,8 +4,9 @@ Date: 2026-08-16
|
||||
Exact base: `e8ac3b65670d696ed5b68f06adf14706bc2ff865`
|
||||
Exact core implementation and local evidence head:
|
||||
`77adf119f871ba820f1f02d55fe2760aa24c84b2`
|
||||
Status: PASS locally; exact-head PR audit, hosted matrix, and canonical merge
|
||||
remain pending
|
||||
Exact canonical head: `9683e0be4d892e6a6a134b45a065e591e139da3c`
|
||||
Canonical run: https://github.com/elder-plinius/OBLITERATUS/actions/runs/31941334488
|
||||
Status: PASS; PR #108 was exact-head audited and canonically merged
|
||||
|
||||
## Results
|
||||
|
||||
@@ -26,9 +27,12 @@ remain pending
|
||||
| Repeatability | 669 tests passed in each of three file orders/hash seeds; 35.353s total; no flake candidates or consistent failures |
|
||||
| Policy | Ruff F, conditional policy, source-to-test risk map, mutation-target guard, global/mature coverage, changed-line, and touched-module gates passed |
|
||||
|
||||
The repository statement and branch exits now both exceed the Gate 3 targets.
|
||||
The remaining new-work hold is bounded item 7, followed by conditional-evidence
|
||||
reconciliation and final canonical reporting.
|
||||
The repository statement and branch exits both exceed the Gate 3 targets. The
|
||||
canonical run passed all eight required checks, including 2,059 tests on each
|
||||
of Python 3.10, 3.11, and 3.12; 89.50% selected mutation score (1,816/2,029);
|
||||
three repeat orders with 669 tests each and no flakes; Windows checkpoint,
|
||||
package, and supply-chain evidence. The remaining new-work hold is bounded item
|
||||
7, followed by conditional-evidence reconciliation and final reporting.
|
||||
|
||||
## Contracts added
|
||||
|
||||
@@ -101,12 +105,12 @@ root/model-directory resolution, raising the final changed-line result to
|
||||
- Mutation metadata and prepared selection evidence:
|
||||
ignored worktree-local `mutants/` artifacts at the exact implementation head
|
||||
|
||||
The exact implementation head passes the enforced Ruff F scope, immutable
|
||||
quality policy, conditional-policy validation, risk-map validation, mutation
|
||||
required-target guard, 75% statement and 60% branch floors, 95% changed-line
|
||||
floor, and touched-module no-regression gate. The full 2,029-mutant campaign,
|
||||
package matrix, supply-chain checks, exact-head audit, and canonical evidence
|
||||
remain hosted PR/merge gates.
|
||||
The exact implementation and canonical heads pass the enforced Ruff F scope,
|
||||
immutable quality policy, conditional-policy validation, risk-map validation,
|
||||
mutation required-target guard, 75% statement and 60% branch floors, 95%
|
||||
changed-line floor, and touched-module no-regression gate. The hosted
|
||||
2,029-mutant campaign, package matrix, supply-chain checks, exact-head audit,
|
||||
and canonical reconciliation are complete.
|
||||
|
||||
## Scope and release decision
|
||||
|
||||
|
||||
@@ -0,0 +1,120 @@
|
||||
# Gate 3 increment 7 tiny-runtime and quantized-semantics report
|
||||
|
||||
Date: 2026-08-16
|
||||
Exact base: `9683e0be4d892e6a6a134b45a065e591e139da3c`
|
||||
Exact implementation head: `acc6b3b254e6d358ddabe495b1ca79295dd36d70`
|
||||
Status: PASS locally; signed exact-head PR audit, hosted matrix, and canonical
|
||||
merge remain pending
|
||||
|
||||
## Results
|
||||
|
||||
| Gate | Result |
|
||||
|---|---|
|
||||
| Focused contracts | 267 restoration, runtime, offline-model, and policy tests passed; the final runtime set is 111/111 green |
|
||||
| Mandatory CPU lane | Python 3.12: 2,112 passed, 9 conditional tests deselected, 0 failed, and 0 skipped in 87.70s |
|
||||
| Repository coverage | 82.76% statements / 69.55% branches before; 83.50% / 71.08% after |
|
||||
| Mature CPU scope | 93.27% / 83.06% before; 94.02% statements / 84.54% branches after |
|
||||
| Immutable mature floor | Raised from 92% / 80% to 94% / 84% in policy and validator |
|
||||
| Changed executable lines | 58/58 (100%) against the exact base before documentation-only changes |
|
||||
| Core pipeline | `abliterate.py`: 50.91% / 38.03% before; 52.63% statements / 40.73% branches after |
|
||||
| Adaptive defaults | 93.59% / 81.65% before; 100% statements / 89.87% branches after |
|
||||
| Advanced metrics | 91.73% / 82.19% before; 100% statements / 98.61% branches after |
|
||||
| Loader decisions | 92.14% / 90.00% before; 100% statements / 96.67% branches after |
|
||||
| Runtime contracts | 100% statements / 100% branches, including the new restoration policy |
|
||||
| Selected mutation | 1,844/2,057 killed (89.65%); 208 survived and 5 timed out outside changed logic; every restoration-helper mutant killed |
|
||||
| Repeatability | 702 tests passed in each of three file orders/hash seeds in 36.857s; no flakes, skips, or consistent failures |
|
||||
| Package | Wheel and sdist built at version 0.1.2; both pass Twine metadata validation |
|
||||
| Policy | Ruff, immutable quality policy, conditional policy, risk map, mutation-target guard, global/mature coverage, and changed-line gates passed |
|
||||
|
||||
The item-7 candidate reaches every quantitative Gate 3 repository and mature
|
||||
CPU exit criterion. Ordinary feature work remains held until this increment is
|
||||
merged and the conditional-evidence and final-reporting items are reconciled.
|
||||
|
||||
## Contracts added
|
||||
|
||||
1. Multi-direction norm restoration measures logical dequantized weights,
|
||||
restores once after the full subspace projection, caps amplification, and
|
||||
fails closed for non-finite or degenerate norms.
|
||||
2. Integer storage is promoted to a logical float parameter instead of casting
|
||||
a projected float back to Byte/uint8 and erasing the result. The exact
|
||||
Float-to-integer failure reported in issue #11 is covered.
|
||||
3. Float and integer restoration preserve `Parameter` identity. Quantized
|
||||
replacement rebinds every module alias to the same replacement parameter,
|
||||
preserving tied/shared-weight semantics.
|
||||
4. Packed modules use their repacker when available, explicitly materialize a
|
||||
float weight with a warning when safe, and fail closed when they expose
|
||||
neither a repacker nor writable logical weight.
|
||||
5. A deterministic, generated GPT-2 fixture executes the real advanced
|
||||
two-direction pipeline, saves and reloads the checkpoint, proves weights
|
||||
changed, and independently verifies every transformer-layer matrix norm.
|
||||
6. Adaptive-default knowledge lookup and merge boundaries, advanced-metric
|
||||
tensor/refusal/KL/CKA boundaries, and loader import-compatibility failures
|
||||
now have executable negative contracts rather than uncovered decision paths.
|
||||
7. The source-to-test risk map now makes projection-math contracts mandatory
|
||||
whenever the core abliteration pipeline changes.
|
||||
|
||||
## Defects detected and corrected
|
||||
|
||||
The pre-change norm restoration iterated raw parameters, converted integer
|
||||
weights to float for arithmetic, and copied the result back to the original
|
||||
integer dtype. That reproduces the reported PyTorch Float-to-Byte failure or,
|
||||
where a cast succeeds, silently destroys fractional projection and rescaling
|
||||
values. It also measured packed bytes instead of logical dequantized weights.
|
||||
|
||||
The corrected path classifies storage before restoration, uses the model's
|
||||
dequantization and replacement seams, and keeps integer values in float storage
|
||||
when no quantization scale/zero-point contract exists. The first tied-weight
|
||||
review then exposed a second defect: replacing one quantized parameter could
|
||||
break aliases. Alias rebinding now preserves the tie after replacement.
|
||||
|
||||
The changed-line gate initially reported 43/47 covered lines. Packed fallback,
|
||||
degenerate no-op, and in-place float restoration contracts raised the final
|
||||
production result to 58/58. Coverage review also found a duplicate unreachable
|
||||
empty-sequence check in first-token KL validation; the shared tensor validator
|
||||
already rejects that input, so the dead branch was removed.
|
||||
|
||||
## Verification and evidence
|
||||
|
||||
- Mandatory Python 3.12 coverage:
|
||||
`/tmp/obliteratus-item7-final2-coverage-py3.12.json`
|
||||
- Mandatory Python 3.12 JUnit:
|
||||
`/tmp/obliteratus-item7-final2-junit-py3.12.xml`
|
||||
- Exact-base Python 3.12 coverage:
|
||||
`/tmp/obliteratus-main-9683e0b-zvr8hF/test-evidence-py3.12/coverage-py3.12.json`
|
||||
- Clean non-editable Python 3.12 test environment:
|
||||
`/tmp/obliteratus-item7-py312-env-AxUcVD`
|
||||
- Mutation artifacts:
|
||||
ignored worktree-local `mutants/` evidence prepared from the exact candidate;
|
||||
final score 89.65% (1,844/2,057) with no changed-helper survivor
|
||||
- Repeat evidence: `/tmp/obliteratus-item7-repeat-gate.json`
|
||||
- Package artifacts: `/tmp/obliteratus-item7-package-QBOhVU/`; SHA-256
|
||||
`34dcd464df0661bf63da3a9c4e406b7acbfa73e6fd09127aef6211d7a30a249e`
|
||||
(wheel) and
|
||||
`f487391c98e4931896238b38b5232fb2f7c08147ec90767328734bf04c5c5c69`
|
||||
(sdist)
|
||||
|
||||
The first isolated sync selected unsupported Python 3.13 and was not used as
|
||||
evidence. A separate reused Python 3.12 environment correctly failed two
|
||||
installed-artifact provenance assertions because it imported a different
|
||||
worktree. The final environment was recreated non-editably from this worktree,
|
||||
its isolated import resolved to its own `site-packages`, and all 2,112 selected
|
||||
tests passed.
|
||||
|
||||
## Signing and integration policy
|
||||
|
||||
Every candidate commit is signed with the configured repository publish key
|
||||
fingerprint `62297562B1C7053088F405DB0117DAAA677A5BF2` through the approved host
|
||||
GPG wrapper and verified locally with `git verify-commit`. Integration must
|
||||
preserve that signature by exact fast-forward or create a locally signed merge
|
||||
commit. GitHub rebase merge is not acceptable because it recreates unsigned
|
||||
canonical commits.
|
||||
|
||||
## Scope and release decision
|
||||
|
||||
This increment uses a synthetic random-initialized offline model and isolated
|
||||
temporary filesystems. It makes no network, external-model, accelerator,
|
||||
credential, or publication claim. The exact-head PR must still pass all hosted
|
||||
CPU, repeat, mutation, package, Windows, and supply-chain gates before a
|
||||
signature-preserving merge. Conditional evidence reconciliation and final Gate
|
||||
3 reporting remain open after merge, so ordinary feature work does not resume
|
||||
yet.
|
||||
@@ -299,12 +299,12 @@ resistance, environment evidence, or deterministic replay.
|
||||
|
||||
| Measure | Current | Gate 3 minimum |
|
||||
|---|---:|---:|
|
||||
| Repository statements | 82.78% item-6 candidate | 80.00% |
|
||||
| Repository branches | 69.62% item-6 candidate | 65.00% |
|
||||
| Mature CPU statements | 93.27% item-6 candidate; 73 lines remain | 94.00% |
|
||||
| Mature CPU branches | 83.06% item-6 candidate; 29 branches remain | 84.00% |
|
||||
| Repository statements | 83.50% item-7 candidate | 80.00% |
|
||||
| Repository branches | 71.08% item-7 candidate | 65.00% |
|
||||
| Mature CPU statements | 94.02% item-7 candidate | 94.00% |
|
||||
| Mature CPU branches | 84.54% item-7 candidate | 84.00% |
|
||||
| Changed executable lines | 95.00% floor | 95.00% floor |
|
||||
| Selected mutation score | 89.46% item-5 canonical; item-6 campaign expands to 2,029 mutants | at least 85% on expanded scope |
|
||||
| Selected mutation score | 89.65% item-7 candidate; 1,844/2,057 killed | at least 85% on expanded scope |
|
||||
| Unexpected warnings | 0 | 0 |
|
||||
| Active flaky quarantines | 0 | 0 |
|
||||
| Software conditional evidence age | 1 day at the item-6 candidate date | at most 8 days |
|
||||
@@ -395,11 +395,16 @@ single repository-wide coverage PR.
|
||||
5. **Completed at `aa182cc` (PR #105).** Add BESTIARY/model-client/watchtower
|
||||
transport and scheduler state contracts. Exact evidence is recorded in
|
||||
`gate3-increment-5-report.md` and the canonical PR follow-up.
|
||||
6. **Implemented locally at `77adf11`.** Add tournament, interactive, and UI
|
||||
decision-seam contracts. Exact local evidence is recorded in
|
||||
`gate3-increment-6-report.md`; audit and canonical merge remain pending.
|
||||
7. Extend the installed tiny-model vertical slice and quantized-storage
|
||||
semantics, including the exact Float-to-integer restoration regression.
|
||||
6. **Completed at `9683e0b` (PR #108).** Add tournament, interactive, and UI
|
||||
decision-seam contracts. Canonical run 31941334488 passed all eight required
|
||||
checks; exact evidence is recorded in `gate3-increment-6-report.md` and the
|
||||
canonical PR reconciliation comment.
|
||||
7. **Implemented locally at `acc6b3b` against `9683e0b`.** Extend the installed
|
||||
tiny-model vertical slice and quantized-storage semantics, including the
|
||||
exact Float-to-integer restoration regression. The candidate reaches 94.02%
|
||||
mature CPU statements and 84.54% branches and raises the immutable floors
|
||||
to 94% / 84%; signed commit, exact-head audit, hosted checks, and canonical
|
||||
merge remain pending.
|
||||
8. Reconcile conditional software evidence, then run or explicitly waive each
|
||||
unavailable hardware/remote environment under the claim policy.
|
||||
9. Publish the Gate 3 execution, coverage, mutation, repeat, duration, and
|
||||
|
||||
Reference in New Issue
Block a user