diff --git a/.aiwg/testing/gate3-increment-5-report.md b/.aiwg/testing/gate3-increment-5-report.md index 8f43407..cdd0822 100644 --- a/.aiwg/testing/gate3-increment-5-report.md +++ b/.aiwg/testing/gate3-increment-5-report.md @@ -3,7 +3,7 @@ Date: 2026-08-16 Exact base: `d7d2ad566af778b1d315616bbea8b8a3c4dd2191` Exact core implementation and evidence head: -`f00878768e00ea02f5b4c9d1bb4bd9d8a9cef730` +`2a7b3818c89a017c152f9f787c876891cc89e648` Status: PASS locally; exact-head PR audit, hosted matrix, and canonical merge remain pending @@ -11,26 +11,26 @@ remain pending | Gate | Result | |---|---| -| Focused service contracts | 97 BESTIARY, model-client, and Watchtower tests; slowest affected test 0.009s and combined affected-test time 0.526s | -| Mandatory CPU lane | Python 3.12: 1,980 passed in 82.63s; 9 conditional tests deselected by policy; 0 failures, errors, skips, or unexpected warnings | +| Focused service contracts | 98 BESTIARY, model-client, and Watchtower tests; slowest affected test 0.060s and combined affected-test time 0.541s | +| Mandatory CPU lane | Python 3.12: 1,981 passed in 79.28s; 9 conditional tests deselected by policy; 0 failures, errors, skips, or unexpected warnings | | Repository coverage | 76.76% statements / 63.49% branches before; 78.69% / 66.05% after | | Mature CPU scope | 93.20% statements / 82.86% branches before; 93.26% / 83.05% after | -| Changed executable lines | 324/324 (100%) against the exact base | +| Changed executable lines | 328/328 (100%) against the exact base | | Touched production modules | 4/4 pass exact-base line and branch no-regression | | BESTIARY adapter | `bestiary_sync.py`: 96.55% statements / 100% branches, from 0% / 0% | | Catalog client | `models_client.py`: 92.86% statements / 92.00% branches, from 27.12% / 3.57% | -| Watchtower | `watchtower.py`: 96.97% statements / 92.22% branches, from 24.63% / 0% | +| Watchtower | `watchtower.py`: 96.99% statements / 92.22% branches, from 24.63% / 0% | | Pure service seam | `service_contracts.py`: 100% statements / 95.65% branches | | Selective mutation | 1,807/2,020 killed (89.46%); 208 survived; 5 timed out; 0 no-test, skipped, suspicious, interrupted, or segfault results | | Changed service mutation | 189/189 killed; 0 survived or timed out; required-target guard passed | -| Repeatability | 594 tests passed in each of three file orders/hash seeds; 33.504s total; no flake candidates or consistent failures | +| Repeatability | 595 tests passed in each of three file orders/hash seeds; 29.259s total; no flake candidates or consistent failures | | Package | sdist and wheel built; required modules and console entry point present; isolated Python 3.12 wheel import/version contract passed | | Policy and workflow | Locked dependency graph, Ruff F, actionlint 1.7.12, conditional policy, risk map, supply-chain policy, and 158 CI/policy contract tests passed | -The final mandatory lane added 97 passing tests relative to the canonical -increment-4 lane while completing 2.21 seconds faster (82.63s versus 84.84s). -The affected service tests consume 0.526s in aggregate, and no affected test -exceeds 0.009s. The repository branch-coverage Gate 3 exit of 65% is now met; +The final mandatory lane added 98 passing tests relative to the canonical +increment-4 lane while completing 5.56 seconds faster (79.28s versus 84.84s). +The affected service tests consume 0.541s in aggregate, and no affected test +exceeds 0.060s. The repository branch-coverage Gate 3 exit of 65% is now met; repository statements and mature-scope exits remain assigned to increments 6–7. @@ -59,7 +59,9 @@ repository statements and mature-scope exits remain assigned to increments recognized by name tokens or exact tags rather than unsafe substrings. 8. Persisted Watchtower state loads transactionally: a malformed envelope, key/ID mismatch, field type, count, status, or metrics object cannot leave a - partially accepted in-memory state. Failed temporary writes are cleaned up. + partially accepted in-memory state. Failed temporary writes are cleaned up, + and each in-process snapshot/write/replace sequence owns the shared + temporary path until promotion completes. 9. Scheduler interval and status inputs are validated by pure contracts. Start, restart, stop, timeout, and start-failure transitions are explicit; a stuck thread remains owned and blocks a duplicate scheduler. @@ -83,6 +85,12 @@ generated sandbox copy after these tests changed; rebuilding only the ignored sandbox artifacts restored the exact 2,020-mutant campaign without touching repository source. +The first exact-SHA audit found one further data-integrity gap: concurrent +scan/status saves could snapshot under the lock and then race on the shared +temporary file after releasing it. A controlled overlapping-writer test now +proves that snapshot, write, and replacement are serialized; related statistics +are captured under the same state lock. + The contracts additionally detect unbounded or ambiguous catalog resolution, malformed records, case-sensitive filter drift, curated-preset replacement, license/instruction substring false positives, non-deterministic organization @@ -92,15 +100,15 @@ background threads, and lost ownership after stop timeout. ## Verification and evidence - Exact-head normalized JUnit, coverage, repeat, mutation, and trend evidence: - `/tmp/obliteratus-item5-final-evidence-f008787/` + `/tmp/obliteratus-item5-final-evidence-2a7b381/` - Exact-head Python 3.12 raw coverage/JUnit evidence: - `/tmp/obliteratus-item5-final-py312-f008787/` + `/tmp/obliteratus-item5-final-py312-2a7b381/` - Exact-base canonical Python 3.12 evidence: `/tmp/obliteratus-postmerge-item4-py312-CH4T03/` - Exact-head package artifacts and hashes: - `/tmp/obliteratus-item5-package-f008787/` + `/tmp/obliteratus-item5-package-2a7b381/` - Installed-wheel verification environment: - `/tmp/obliteratus-item5-wheel-env-f008787/` + `/tmp/obliteratus-item5-wheel-env-2a7b381/` The exact-head Ruff F gate, recovered pinned actionlint, `uv lock --check`, conditional-policy validation, risk-map validation, changed-line gate, diff --git a/.aiwg/testing/master-test-plan.md b/.aiwg/testing/master-test-plan.md index dad2f3b..c1d0933 100644 --- a/.aiwg/testing/master-test-plan.md +++ b/.aiwg/testing/master-test-plan.md @@ -28,11 +28,11 @@ supply-chain, and all other hosted jobs passed on attempt 1: https://github.com/elder-plinius/OBLITERATUS/actions/runs/31926709748. Gate 3 increment 5 is implemented and locally verified at exact core evidence -head `f00878768e00ea02f5b4c9d1bb4bd9d8a9cef730`. Python 3.12 passes 1,980 -mandatory tests in 82.63s; coverage is 78.69% statements / 66.05% branches; -mature CPU coverage is 93.26% / 83.05%; 324/324 changed executable lines and +head `2a7b3818c89a017c152f9f787c876891cc89e648`. Python 3.12 passes 1,981 +mandatory tests in 79.28s; coverage is 78.69% statements / 66.05% branches; +mature CPU coverage is 93.26% / 83.05%; 328/328 changed executable lines and all four touched modules pass. The expanded campaign kills 1,807 of 2,020 -mutants (89.46%), including all 189 new service-state mutants, and 594 tests +mutants (89.46%), including all 189 new service-state mutants, and 595 tests pass in each repeat order. Exact-head PR audit, hosted matrix, and canonical merge remain pending; details are in `gate3-increment-5-report.md`. @@ -359,7 +359,7 @@ small pure seam and measure it rather than mocking away the behavior under test. | `tourney.py` | 12.01% / 0% | at least 55% / 40% for lifecycle/state transitions | | `bestiary_sync.py` | 96.55% / 100% item-5 candidate | at least 80% / 65% using a loopback/fake transport | | `models_client.py` | 92.86% / 92.00% item-5 candidate | at least 80% / 65% for resolution/error contracts | -| `watchtower.py` | 96.97% / 92.22% item-5 candidate | at least 70% / 55% with injected clock/client seams | +| `watchtower.py` | 96.99% / 92.22% item-5 candidate | at least 70% / 55% with injected clock/client seams | | `interactive.py` | 10.49% / 0% | at least 70% / 55% with scripted I/O contracts | | `local_ui.py` | 48.15% / 15.22% | at least 70% / 55% for launch decisions and failures | | `ui_watchtower.py` | 0% / 0% | cover construction/callback contracts; real UI stays conditional | @@ -385,7 +385,7 @@ single repository-wide coverage PR. concurrency, atomicity, retry, cleanup-ownership, and Windows portability contracts. Canonical evidence is recorded in `gate3-increment-4-report.md`. -5. **Implemented locally at `f008787`.** Add BESTIARY/model-client/watchtower +5. **Implemented locally at `2a7b381`.** Add BESTIARY/model-client/watchtower transport and scheduler state contracts. Exact evidence is recorded in `gate3-increment-5-report.md`; audit and canonical merge remain pending. 6. Add tournament, interactive, and UI decision-seam contracts.