mirror of
https://github.com/elder-plinius/OBLITERATUS.git
synced 2026-08-17 16:37:30 +02:00
docs: record Gate 3 operator interface evidence
This commit is contained in:
Joseph Magly
@@ -4,8 +4,8 @@ Date: 2026-08-16
|
||||
Exact base: `d7d2ad566af778b1d315616bbea8b8a3c4dd2191`
|
||||
Exact core implementation and evidence head:
|
||||
`2a7b3818c89a017c152f9f787c876891cc89e648`
|
||||
Status: PASS locally; exact-head PR audit, hosted matrix, and canonical merge
|
||||
remain pending
|
||||
Canonical commit: `aa182cc44883890c019c393f17602e2e2702e7d7`
|
||||
Status: PASS canonical (PR #105, rebase merged)
|
||||
|
||||
## Results
|
||||
|
||||
@@ -109,13 +109,19 @@ background threads, and lost ownership after stop timeout.
|
||||
`/tmp/obliteratus-item5-package-2a7b381/`
|
||||
- Installed-wheel verification environment:
|
||||
`/tmp/obliteratus-item5-wheel-env-2a7b381/`
|
||||
- Canonical 8/8 hosted evidence:
|
||||
https://github.com/elder-plinius/OBLITERATUS/actions/runs/31933066525
|
||||
|
||||
The exact-head Ruff F gate, recovered pinned actionlint, `uv lock --check`,
|
||||
conditional-policy validation, risk-map validation, changed-line gate,
|
||||
touched-module gate, immutable quality policy, duration policy, repeat gate,
|
||||
mutation score and required-target gates, package checks, and supply-chain
|
||||
policy all pass. The original operator checkout was not modified; work and
|
||||
evidence remain isolated in the dedicated item-5 worktree and temporary paths.
|
||||
policy all pass. The canonical run completed all eight jobs successfully; its
|
||||
1,805/2,020 mutation result differed from the PR/local 1,807/2,020 result only
|
||||
because two simple mutants timed out on that runner and were killed on the
|
||||
identical tree in both other campaigns. The original operator checkout was not
|
||||
modified; work and evidence remained isolated in the dedicated item-5 worktree
|
||||
and temporary paths.
|
||||
|
||||
## Scope and release decision
|
||||
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
# Gate 3 increment 6 tournament and operator-interface report
|
||||
|
||||
Date: 2026-08-16
|
||||
Exact base: `e8ac3b65670d696ed5b68f06adf14706bc2ff865`
|
||||
Exact core implementation and local evidence head:
|
||||
`8a13ea9965c0c455b42262c3b81e4ea3bf9ae798`
|
||||
Status: PASS locally; exact-head PR audit, hosted matrix, and canonical merge
|
||||
remain pending
|
||||
|
||||
## Results
|
||||
|
||||
| Gate | Result |
|
||||
|---|---|
|
||||
| Focused contracts | 70 new tournament, interactive, local-launcher, and Watchtower-UI tests passed |
|
||||
| Mandatory CPU lane | Python 3.12: 2,053 passed, 2 expected source-shadow xfails, and 9 conditional tests deselected in 56.34s |
|
||||
| Repository coverage | 78.69% statements / 66.05% branches before; 82.77% / 69.60% after |
|
||||
| Mature CPU scope | 93.27% statements / 83.06% branches |
|
||||
| Changed executable lines | 51/51 (100%) against the exact base |
|
||||
| Touched production modules | 5/5 pass exact-base line and branch no-regression |
|
||||
| Tournament lifecycle | `tourney.py`: 12.01% / 0% before; 83.09% statements / 77.78% branches after |
|
||||
| Interactive decisions | `interactive.py`: 10.49% / 0% before; 76.92% / 73.53% after |
|
||||
| Local launcher | `local_ui.py`: 48.15% / 15.22% before; 85.19% / 63.04% after |
|
||||
| Watchtower UI handlers | `ui_watchtower.py`: 0% / 0% before; 70.89% / 83.33% after |
|
||||
| Pure checkpoint seam | `tourney_contracts.py`: 100% statements / 100% branches |
|
||||
| Changed-seam mutation | 9/9 checkpoint-parser mutants killed; configured campaign expands from 2,020 to 2,029 mutants |
|
||||
| Repeatability | 665 tests passed in each of three file orders/hash seeds; 34.300s total; no flake candidates or consistent failures |
|
||||
| Policy | Ruff F, conditional policy, source-to-test risk map, mutation-target guard, global/mature coverage, changed-line, and touched-module gates passed |
|
||||
|
||||
The repository statement and branch exits now both exceed the Gate 3 targets.
|
||||
The remaining new-work hold is bounded item 7, followed by conditional-evidence
|
||||
reconciliation and final canonical reporting.
|
||||
|
||||
## Contracts added
|
||||
|
||||
1. Tournament scoring, result ordering, bracket/model-card rendering, round
|
||||
ranking, loser cleanup, no-winner behavior, and callback sequencing are
|
||||
deterministic without loading a model.
|
||||
2. Checkpoints reject malformed, non-object, unreadable, unsupported-version,
|
||||
and invalid-encoding documents. Completed and interrupted rounds preserve
|
||||
direction and spectral-certification metadata, and quota interruption resumes
|
||||
at the exact remaining method without repeating completed work.
|
||||
3. Guided terminal choices map hardware boundaries, custom models, presets,
|
||||
strategies, sample sizes, cancellation, and quantized execution into an exact
|
||||
study configuration. Remote repository code is disabled by default, and the
|
||||
quantized branch returns its study result.
|
||||
4. The local launcher masks passwords, reports network listeners, forwards the
|
||||
full Gradio server contract, avoids hardware work in quiet mode, fails clearly
|
||||
when the optional UI runtime is absent, and degrades to CPU when a GPU driver
|
||||
probe raises a runtime error.
|
||||
5. Watchtower pure handlers are mandatory even without Gradio. Actual tab
|
||||
construction still requires the optional UI extra, while update mappings,
|
||||
scan/history/scheduler callbacks, zero-valued metrics, and failure rendering
|
||||
remain CPU-testable.
|
||||
6. One-click result selection orders numeric iteration identifiers correctly,
|
||||
ignores malformed names and unreadable entries, and refuses symlink targets
|
||||
outside its managed result root.
|
||||
7. Dynamic Watchtower status and scan errors are HTML-escaped, and an exact
|
||||
`0.0` refusal rate is reported as measured zero rather than `unknown`.
|
||||
|
||||
## Defects detected and corrected
|
||||
|
||||
The first focused tournament run found that non-object JSON roots raised
|
||||
`AttributeError` and that interrupted contenders lost direction/certification
|
||||
metadata on resume. Scripted interactive tests then exposed unconditional
|
||||
`trust_remote_code=True` and a discarded quantized-study return value. Launcher
|
||||
tests reproduced a driver-probe exception that prevented CPU fallback.
|
||||
|
||||
Watchtower handler tests detected lexicographic iteration ordering (`iter_2`
|
||||
beating `iter_10`), symlink escape from the managed download root, unescaped
|
||||
HTML in scan errors and invalid timestamps, and false `unknown` reporting for a
|
||||
measured zero refusal rate. The initial mandatory run also proved that a
|
||||
collection-time Gradio stub contaminated conditional test selection; pure
|
||||
handlers now have a real optional-dependency seam instead.
|
||||
|
||||
The first changed-line comparison reported 35/40 covered executable changes.
|
||||
Negative tests for the installed-Gradio update path, empty model IDs, malformed
|
||||
iteration names, and unreadable iteration directories raised the final gate to
|
||||
51/51 without weakening the 95% policy.
|
||||
|
||||
## Verification and evidence
|
||||
|
||||
- Mandatory Python 3.12 coverage:
|
||||
`/tmp/obliteratus-item6-rebased-coverage.json`
|
||||
- Mandatory Python 3.12 JUnit:
|
||||
`/tmp/obliteratus-item6-rebased-junit.xml`
|
||||
- Exact-base Python 3.12 coverage:
|
||||
`/tmp/obliteratus-main-e8ac3b6-UGNOIz/coverage-py3.12.json`
|
||||
- Final repeat evidence:
|
||||
`/tmp/obliteratus-item6-rebased-repeat-vU1gQh/`
|
||||
- Exact-head package artifacts:
|
||||
`/tmp/obliteratus-item6-rebased-package-tRW7rz/`
|
||||
- Mutation metadata and prepared selection evidence:
|
||||
ignored worktree-local `mutants/` artifacts at the exact implementation head
|
||||
|
||||
The exact implementation head passes the enforced Ruff F scope, immutable
|
||||
quality policy, conditional-policy validation, risk-map validation, mutation
|
||||
required-target guard, 75% statement and 60% branch floors, 95% changed-line
|
||||
floor, and touched-module no-regression gate. The full 2,029-mutant campaign,
|
||||
package matrix, supply-chain checks, exact-head audit, and canonical evidence
|
||||
remain hosted PR/merge gates.
|
||||
|
||||
## Scope and release decision
|
||||
|
||||
This increment implements bounded delivery item 6. It makes no live model,
|
||||
network, browser, accelerator, credential, or publication claim. All model and
|
||||
service behavior uses injected fakes and isolated temporary filesystems. Item 7
|
||||
and the final Gate 3 release decision remain open, so ordinary feature work does
|
||||
not resume yet.
|
||||
@@ -18,23 +18,30 @@ Gate 3 increment 3 canonical commit:
|
||||
Gate 3 increment 4 canonical commit:
|
||||
`d7d2ad566af778b1d315616bbea8b8a3c4dd2191` (PR #104, rebase merged)
|
||||
|
||||
The latest canonical mandatory offline CPU lane passes 1,883 tests on Python
|
||||
3.12 with 9 conditional tests deselected by policy, no failures or warnings,
|
||||
and 76.76% statement / 63.49% branch coverage. Mature CPU-testable coverage is
|
||||
93.20% / 82.86%. The canonical mutation campaign kills 1,618 of 1,831 mutants
|
||||
(88.37%), including every changed persistence mutant, while 497 tests pass in
|
||||
each of three file orders and hash seeds. Package, Windows checkpoint,
|
||||
supply-chain, and all other hosted jobs passed on attempt 1:
|
||||
https://github.com/elder-plinius/OBLITERATUS/actions/runs/31926709748.
|
||||
Gate 3 increment 5 canonical commit:
|
||||
`aa182cc44883890c019c393f17602e2e2702e7d7` (PR #105, rebase merged)
|
||||
|
||||
Gate 3 increment 5 is implemented and locally verified at exact core evidence
|
||||
head `2a7b3818c89a017c152f9f787c876891cc89e648`. Python 3.12 passes 1,981
|
||||
mandatory tests in 79.28s; coverage is 78.69% statements / 66.05% branches;
|
||||
mature CPU coverage is 93.26% / 83.05%; 328/328 changed executable lines and
|
||||
all four touched modules pass. The expanded campaign kills 1,807 of 2,020
|
||||
mutants (89.46%), including all 189 new service-state mutants, and 595 tests
|
||||
pass in each repeat order. Exact-head PR audit, hosted matrix, and canonical
|
||||
merge remain pending; details are in `gate3-increment-5-report.md`.
|
||||
Current canonical AIWG and CI-contract baseline:
|
||||
`e8ac3b65670d696ed5b68f06adf14706bc2ff865` (PR #107, rebase merged)
|
||||
|
||||
The latest canonical mandatory offline CPU lane passes 1,985 tests on Python
|
||||
3.12 with 9 conditional tests deselected by policy, no failures, errors, skips,
|
||||
or unexpected warnings, and 78.57% statement / 65.89% branch coverage. Mature
|
||||
CPU-testable coverage is 93.09% / 82.89%. The canonical mutation campaign kills
|
||||
1,807 of 2,020 mutants (89.46%), with 208 survivors and 5 timeouts, while 595
|
||||
tests pass in each of three file orders and hash seeds with no flakes. Package,
|
||||
Windows checkpoint, supply-chain, and all other hosted jobs passed on attempt 1:
|
||||
https://github.com/elder-plinius/OBLITERATUS/actions/runs/31937799938.
|
||||
|
||||
Gate 3 increment 6 is implemented and locally verified at exact core evidence
|
||||
head `8a13ea9965c0c455b42262c3b81e4ea3bf9ae798`. Python 3.12 passes 2,053
|
||||
mandatory tests with 2 expected source-shadow xfails and 9 conditional tests
|
||||
deselected in 56.34s; coverage is 82.77% statements / 69.60% branches; mature
|
||||
CPU coverage is 93.27% / 83.06%; 51/51 changed executable lines and all five
|
||||
touched modules pass. All 9 new checkpoint-parser mutants are killed, the full
|
||||
campaign expands to 2,029 mutants for hosted verification, and 665 tests pass
|
||||
in each repeat order with no flakes. Exact-head PR audit, hosted matrix, and
|
||||
canonical merge remain pending; details are in `gate3-increment-6-report.md`.
|
||||
|
||||
The latest conditional run is green for pinned-model download and evaluation,
|
||||
loopback network, operator UI, and evidence-freshness policy at the earlier
|
||||
@@ -51,9 +58,9 @@ property, mutation, determinism, and real-environment evidence without weakening
|
||||
the mandatory CPU baseline.
|
||||
|
||||
The operator has now placed ordinary feature work back behind a testing-depth
|
||||
gate. Gate 3 is adopted and items 1–4 are canonical; item 5 is locally complete
|
||||
but not yet canonical; items 6–7 and the remaining quantitative exit criteria
|
||||
must pass before unrelated feature work resumes.
|
||||
gate. Gate 3 is adopted and items 1–5 are canonical; item 6 is locally complete
|
||||
but not yet canonical; item 7 and the remaining quantitative exit criteria must
|
||||
pass before unrelated feature work resumes.
|
||||
Correctness, security, data-loss, and test-infrastructure repairs remain
|
||||
permitted when narrowly scoped and paired with reproducing tests.
|
||||
|
||||
@@ -292,15 +299,15 @@ resistance, environment evidence, or deterministic replay.
|
||||
|
||||
| Measure | Current | Gate 3 minimum |
|
||||
|---|---:|---:|
|
||||
| Repository statements | 78.69% item-5 candidate | 80.00% |
|
||||
| Repository branches | 66.05% item-5 candidate | 65.00% |
|
||||
| Mature CPU statements | 93.26% item-5 candidate | 94.00% |
|
||||
| Mature CPU branches | 83.05% item-5 candidate | 84.00% |
|
||||
| Repository statements | 82.77% item-6 candidate | 80.00% |
|
||||
| Repository branches | 69.60% item-6 candidate | 65.00% |
|
||||
| Mature CPU statements | 93.27% item-6 candidate; 73 lines remain | 94.00% |
|
||||
| Mature CPU branches | 83.06% item-6 candidate; 29 branches remain | 84.00% |
|
||||
| Changed executable lines | 95.00% floor | 95.00% floor |
|
||||
| Selected mutation score | 89.46% item-5 candidate on expanded service scope | at least 85% on expanded scope |
|
||||
| Selected mutation score | 89.46% item-5 canonical; item-6 campaign expands to 2,029 mutants | at least 85% on expanded scope |
|
||||
| Unexpected warnings | 0 | 0 |
|
||||
| Active flaky quarantines | 0 | 0 |
|
||||
| Software conditional evidence age | at most 8 days | at most 8 days |
|
||||
| Software conditional evidence age | 1 day at the item-6 candidate date | at most 8 days |
|
||||
|
||||
The expanded mutation scope must include core mutation math, architecture and
|
||||
loader decisions, persistence/atomicity, evaluation/report serialization, and
|
||||
@@ -385,10 +392,12 @@ single repository-wide coverage PR.
|
||||
concurrency, atomicity, retry, cleanup-ownership, and Windows portability
|
||||
contracts. Canonical evidence is recorded in
|
||||
`gate3-increment-4-report.md`.
|
||||
5. **Implemented locally at `2a7b381`.** Add BESTIARY/model-client/watchtower
|
||||
5. **Completed at `aa182cc` (PR #105).** Add BESTIARY/model-client/watchtower
|
||||
transport and scheduler state contracts. Exact evidence is recorded in
|
||||
`gate3-increment-5-report.md`; audit and canonical merge remain pending.
|
||||
6. Add tournament, interactive, and UI decision-seam contracts.
|
||||
`gate3-increment-5-report.md` and the canonical PR follow-up.
|
||||
6. **Implemented locally at `8a13ea9`.** Add tournament, interactive, and UI
|
||||
decision-seam contracts. Exact local evidence is recorded in
|
||||
`gate3-increment-6-report.md`; audit and canonical merge remain pending.
|
||||
7. Extend the installed tiny-model vertical slice and quantized-storage
|
||||
semantics, including the exact Float-to-integer restoration regression.
|
||||
8. Reconcile conditional software evidence, then run or explicitly waive each
|
||||
|
||||
Reference in New Issue
Block a user