docs: publish Gate 3 item 5 evidence

.aiwg/testing/gate3-increment-5-report.md

@@ -0,0 +1,119 @@
+# Gate 3 increment 5 service-orchestration report
+
+Date: 2026-08-16
+Exact base: `d7d2ad566af778b1d315616bbea8b8a3c4dd2191`
+Exact core implementation and evidence head:
+`f00878768e00ea02f5b4c9d1bb4bd9d8a9cef730`
+Status: PASS locally; exact-head PR audit, hosted matrix, and canonical merge
+remain pending
+
+## Results
+
+| Gate | Result |
+|---|---|
+| Focused service contracts | 97 BESTIARY, model-client, and Watchtower tests; slowest affected test 0.009s and combined affected-test time 0.526s |
+| Mandatory CPU lane | Python 3.12: 1,980 passed in 82.63s; 9 conditional tests deselected by policy; 0 failures, errors, skips, or unexpected warnings |
+| Repository coverage | 76.76% statements / 63.49% branches before; 78.69% / 66.05% after |
+| Mature CPU scope | 93.20% statements / 82.86% branches before; 93.26% / 83.05% after |
+| Changed executable lines | 324/324 (100%) against the exact base |
+| Touched production modules | 4/4 pass exact-base line and branch no-regression |
+| BESTIARY adapter | `bestiary_sync.py`: 96.55% statements / 100% branches, from 0% / 0% |
+| Catalog client | `models_client.py`: 92.86% statements / 92.00% branches, from 27.12% / 3.57% |
+| Watchtower | `watchtower.py`: 96.97% statements / 92.22% branches, from 24.63% / 0% |
+| Pure service seam | `service_contracts.py`: 100% statements / 95.65% branches |
+| Selective mutation | 1,807/2,020 killed (89.46%); 208 survived; 5 timed out; 0 no-test, skipped, suspicious, interrupted, or segfault results |
+| Changed service mutation | 189/189 killed; 0 survived or timed out; required-target guard passed |
+| Repeatability | 594 tests passed in each of three file orders/hash seeds; 33.504s total; no flake candidates or consistent failures |
+| Package | sdist and wheel built; required modules and console entry point present; isolated Python 3.12 wheel import/version contract passed |
+| Policy and workflow | Locked dependency graph, Ruff F, actionlint 1.7.12, conditional policy, risk map, supply-chain policy, and 158 CI/policy contract tests passed |
+
+The final mandatory lane added 97 passing tests relative to the canonical
+increment-4 lane while completing 2.21 seconds faster (82.63s versus 84.84s).
+The affected service tests consume 0.526s in aggregate, and no affected test
+exceeds 0.009s. The repository branch-coverage Gate 3 exit of 65% is now met;
+repository statements and mature-scope exits remain assigned to increments
+6–7.
+
+## Contracts added
+
+1. An explicitly configured catalog path or URL is authoritative. Missing,
+   oversized, non-UTF-8, malformed JSON, and invalid catalog envelopes fail
+   closed instead of silently falling through to an unrelated local catalog.
+2. Local and URL catalog reads are bounded at 8 MiB, URL reads use a finite
+   timeout, and the stable catalog envelope and records are schema-checked.
+3. Channel, vendor, and capability filters normalize case and whitespace;
+   aliases remain stable; malformed list, vendor, identifier, date, and
+   `open_weight` values are rejected rather than coerced.
+4. Newest-model selection uses an injected UTC clock, validates its day window,
+   falls back from `first_seen` to `released`, skips invalid dates, and sorts by
+   the parsed effective date.
+5. BESTIARY-derived presets preserve curated-ID authority and deterministic
+   de-duplication, validate record fields, infer parameter tiers including MoE
+   and unknown-size boundaries, and fail safe as one empty extension on a
+   malformed catalog.
+6. The Hugging Face adapter is injectable and deterministic. Per-organization
+   failures are isolated, minimum-download filtering is enforced, malformed
+   list/candidate responses are ignored or rejected at explicit boundaries,
+   and no test performs a live service call.
+7. License matching respects identifier boundaries, and instruction tuning is
+   recognized by name tokens or exact tags rather than unsafe substrings.
+8. Persisted Watchtower state loads transactionally: a malformed envelope,
+   key/ID mismatch, field type, count, status, or metrics object cannot leave a
+   partially accepted in-memory state. Failed temporary writes are cleaned up.
+9. Scheduler interval and status inputs are validated by pure contracts. Start,
+   restart, stop, timeout, and start-failure transitions are explicit; a stuck
+   thread remains owned and blocks a duplicate scheduler.
+10. The scheduler runs immediately, stops through an injected event without
+    sleep races, and safely defers confirmation when a callback requests stop
+    from the scheduler's own thread, avoiding self-join and orphaning.
+
+## Defects and mutants detected
+
+The first focused campaign killed 166 of 194 service-contract mutants. The 28
+survivors exposed weak boundaries around catalog field types, day/interval
+validation, candidate defaults, state transitions, license matching, and exact
+diagnostics. Survivor-driven tests and simplification of equivalent defaults
+reduced the final pure seam to 189 meaningful mutants and killed every one.
+
+Manual pre-delivery review then found two uncovered behaviors: string/integer
+`open_weight` values could be truth-coerced, and a discovery callback could ask
+the scheduler thread to join itself. Both now have reproducing tests and
+fail-closed implementations. The fresh mutation prepass also rejected a stale
+generated sandbox copy after these tests changed; rebuilding only the ignored
+sandbox artifacts restored the exact 2,020-mutant campaign without touching
+repository source.
+
+The contracts additionally detect unbounded or ambiguous catalog resolution,
+malformed records, case-sensitive filter drift, curated-preset replacement,
+license/instruction substring false positives, non-deterministic organization
+order, partial state acceptance, invalid lifecycle transitions, duplicate
+background threads, and lost ownership after stop timeout.
+
+## Verification and evidence
+
+- Exact-head normalized JUnit, coverage, repeat, mutation, and trend evidence:
+  `/tmp/obliteratus-item5-final-evidence-f008787/`
+- Exact-head Python 3.12 raw coverage/JUnit evidence:
+  `/tmp/obliteratus-item5-final-py312-f008787/`
+- Exact-base canonical Python 3.12 evidence:
+  `/tmp/obliteratus-postmerge-item4-py312-CH4T03/`
+- Exact-head package artifacts and hashes:
+  `/tmp/obliteratus-item5-package-f008787/`
+- Installed-wheel verification environment:
+  `/tmp/obliteratus-item5-wheel-env-f008787/`
+
+The exact-head Ruff F gate, recovered pinned actionlint, `uv lock --check`,
+conditional-policy validation, risk-map validation, changed-line gate,
+touched-module gate, immutable quality policy, duration policy, repeat gate,
+mutation score and required-target gates, package checks, and supply-chain
+policy all pass. The original operator checkout was not modified; work and
+evidence remain isolated in the dedicated item-5 worktree and temporary paths.
+
+## Scope and release decision
+
+This increment implements Gate 3 bounded delivery item 5. It does not reopen
+ordinary feature work: items 6–7, conditional-evidence reconciliation, and the
+final canonical Gate 3 publication remain open. All catalog and Hugging Face
+service behavior in this increment uses injected adapters, operator-provided
+files, or loopback-style fakes. It makes no new live-network, remote,
+accelerator, model-download, browser, or operator-UI support claim.

.aiwg/testing/master-test-plan.md

@@ -1,6 +1,6 @@
 # OBLITERATUS master test plan
 
-Date: 2026-08-15
+Date: 2026-08-16
 Owner: maintainers
 Phase: construction
 
@@ -15,27 +15,26 @@ Gate 3 increment 2 canonical commit:
 Gate 3 increment 3 canonical commit:
 `c1b34503ddd3cb797e5d70671c47afcabfe73832` (PR #103, rebase merged)
 
-The latest canonical mandatory offline CPU selection passes on Python 3.10–3.12
-with 1,830–1,832 passing tests, 9 conditionally deselected tests, 2 expected
-Python 3.12 xfails, no unexpected skips, and no warnings. Branch-enabled Python
-3.12 coverage is 76.60% statements and 63.25% branches; the documented mature
-CPU-testable scope is 93.12% statements and 82.60% branches. The repository
-enforces 95% changed-line coverage, per-touched-module line and branch
-no-regression, an 85% selective mutation floor, zero unexpected warnings, and
-owned suite/test/marker/repeat duration budgets. The canonical increment-3
-campaign killed 1,424 of 1,637 mutants (86.99%), and 431 high-consequence tests
-passed in each of three file orders and hash seeds. Package, installed
-wheel/sdist, vulnerability, secret, license, and SBOM checks are green. All
-seven jobs in the canonical post-merge run passed:
-https://github.com/elder-plinius/OBLITERATUS/actions/runs/31920984688.
+Gate 3 increment 4 canonical commit:
+`d7d2ad566af778b1d315616bbea8b8a3c4dd2191` (PR #104, rebase merged)
 
-Gate 3 increment 4 is implemented and locally verified at core evidence head
-`9da022e6f3bc73d17fd31d4da70b42e3ca052ba2`, with delivery evidence recorded in
-`gate3-increment-4-report.md`. Candidate coverage is 76.77% statements and
-63.55% branches; mature CPU coverage is 93.19% statements and 82.85% branches;
-the complete campaign kills 1,614 of 1,827 mutants (88.34%), including all
-229 persistence mutants. Exact-head PR audit, the first hosted Windows
-checkpoint contract, and canonical merge remain pending.
+The latest canonical mandatory offline CPU lane passes 1,883 tests on Python
+3.12 with 9 conditional tests deselected by policy, no failures or warnings,
+and 76.76% statement / 63.49% branch coverage. Mature CPU-testable coverage is
+93.20% / 82.86%. The canonical mutation campaign kills 1,618 of 1,831 mutants
+(88.37%), including every changed persistence mutant, while 497 tests pass in
+each of three file orders and hash seeds. Package, Windows checkpoint,
+supply-chain, and all other hosted jobs passed on attempt 1:
+https://github.com/elder-plinius/OBLITERATUS/actions/runs/31926709748.
+
+Gate 3 increment 5 is implemented and locally verified at exact core evidence
+head `f00878768e00ea02f5b4c9d1bb4bd9d8a9cef730`. Python 3.12 passes 1,980
+mandatory tests in 82.63s; coverage is 78.69% statements / 66.05% branches;
+mature CPU coverage is 93.26% / 83.05%; 324/324 changed executable lines and
+all four touched modules pass. The expanded campaign kills 1,807 of 2,020
+mutants (89.46%), including all 189 new service-state mutants, and 594 tests
+pass in each repeat order. Exact-head PR audit, hosted matrix, and canonical
+merge remain pending; details are in `gate3-increment-5-report.md`.
 
 The latest conditional run is green for pinned-model download and evaluation,
 loopback network, operator UI, and evidence-freshness policy at the earlier
@@ -52,9 +51,9 @@ property, mutation, determinism, and real-environment evidence without weakening
 the mandatory CPU baseline.
 
 The operator has now placed ordinary feature work back behind a testing-depth
-gate. Gate 3 is adopted and items 1–3 are complete; item 4 is implemented but
-not yet canonical; items 5–7 and the quantitative exit criteria must pass
-before unrelated feature work resumes.
+gate. Gate 3 is adopted and items 1–4 are canonical; item 5 is locally complete
+but not yet canonical; items 6–7 and the remaining quantitative exit criteria
+must pass before unrelated feature work resumes.
 Correctness, security, data-loss, and test-infrastructure repairs remain
 permitted when narrowly scoped and paired with reproducing tests.
 
@@ -293,12 +292,12 @@ resistance, environment evidence, or deterministic replay.
 
 | Measure | Current | Gate 3 minimum |
 |---|---:|---:|
-| Repository statements | 76.76% canonical | 80.00% |
-| Repository branches | 63.49% canonical | 65.00% |
-| Mature CPU statements | 93.20% canonical | 94.00% |
-| Mature CPU branches | 82.86% canonical | 84.00% |
+| Repository statements | 78.69% item-5 candidate | 80.00% |
+| Repository branches | 66.05% item-5 candidate | 65.00% |
+| Mature CPU statements | 93.26% item-5 candidate | 94.00% |
+| Mature CPU branches | 83.05% item-5 candidate | 84.00% |
 | Changed executable lines | 95.00% floor | 95.00% floor |
-| Selected mutation score | 88.37% canonical on expanded persistence scope | at least 85% on expanded scope |
+| Selected mutation score | 89.46% item-5 candidate on expanded service scope | at least 85% on expanded scope |
 | Unexpected warnings | 0 | 0 |
 | Active flaky quarantines | 0 | 0 |
 | Software conditional evidence age | at most 8 days | at most 8 days |
@@ -358,9 +357,9 @@ small pure seam and measure it rather than mocking away the behavior under test.
 | `abliterate.py` | 50.91% / 38.03% candidate | at least 52% / 45% with mutation/math contracts |
 | `lora_ablation.py` | 9.57% / 0% | at least 70% / 55% for CPU-testable decisions |
 | `tourney.py` | 12.01% / 0% | at least 55% / 40% for lifecycle/state transitions |
-| `bestiary_sync.py` | 0% / 0% | at least 80% / 65% using a loopback/fake transport |
-| `models_client.py` | 27.12% / 3.57% | at least 80% / 65% for resolution/error contracts |
-| `watchtower.py` | 24.63% / 0% | at least 70% / 55% with injected clock/client seams |
+| `bestiary_sync.py` | 96.55% / 100% item-5 candidate | at least 80% / 65% using a loopback/fake transport |
+| `models_client.py` | 92.86% / 92.00% item-5 candidate | at least 80% / 65% for resolution/error contracts |
+| `watchtower.py` | 96.97% / 92.22% item-5 candidate | at least 70% / 55% with injected clock/client seams |
 | `interactive.py` | 10.49% / 0% | at least 70% / 55% with scripted I/O contracts |
 | `local_ui.py` | 48.15% / 15.22% | at least 70% / 55% for launch decisions and failures |
 | `ui_watchtower.py` | 0% / 0% | cover construction/callback contracts; real UI stays conditional |
@@ -386,7 +385,9 @@ single repository-wide coverage PR.
    concurrency, atomicity, retry, cleanup-ownership, and Windows portability
    contracts. Canonical evidence is recorded in
    `gate3-increment-4-report.md`.
-5. Add BESTIARY/model-client/watchtower transport and scheduler state contracts.
+5. **Implemented locally at `f008787`.** Add BESTIARY/model-client/watchtower
+   transport and scheduler state contracts. Exact evidence is recorded in
+   `gate3-increment-5-report.md`; audit and canonical merge remain pending.
 6. Add tournament, interactive, and UI decision-seam contracts.
 7. Extend the installed tiny-model vertical slice and quantized-storage
    semantics, including the exact Float-to-integer restoration regression.