docs: define Gate 3 testing depth plan

.aiwg/testing/master-test-plan.md

@@ -8,6 +8,8 @@ Phase: construction
 
 Gate 2 canonical code commit: `fa99d5a424f8725f6e7e0a97af4f8abd7bcf1f56`
 
+Current post-merge baseline commit: `eab67c993d53b88ed9168096d4de20e8ee43a374`
+
 The mandatory offline CPU selection passes on Python 3.10-3.12 with 1,541
 tests, 9 conditionally deselected tests, no skips, and no warnings. The slowest
 hosted pytest selection is Python 3.12 at 136.33 seconds; its complete CI job is
@@ -28,11 +30,25 @@ https://github.com/elder-plinius/OBLITERATUS/actions/runs/31868394039. CUDA,
 bitsandbytes, MPS, MLX, and remote execution were not selected and therefore
 have no fresh support evidence or support claim.
 
-The foundation, Wave A, Gate 1, and Gate 2 now establish a sane,
-evidence-producing baseline. The contributor merge train and ordinary feature
-work may proceed one exact-head PR at a time under the per-PR test contract.
-Wave C and Wave D continue to deepen property, mutation, determinism, and real
-environment evidence without weakening the mandatory CPU baseline.
+The foundation, Wave A, Gate 1, and Gate 2 establish a sane,
+evidence-producing baseline. The legacy contributor merge train proceeded one
+exact-head PR at a time under the per-PR test contract. Wave C and Wave D deepen
+property, mutation, determinism, and real-environment evidence without weakening
+the mandatory CPU baseline.
+
+After the Gate 2 contributor integrations, the mandatory suite contains 1,585
+passing tests, 9 conditional deselections, and 2 documented expected failures.
+Repository coverage is 75.80% statements and 61.89% branches; mature CPU scope
+is 92.89% statements and 81.72% branches. All seven jobs at the current
+post-merge commit passed:
+https://github.com/elder-plinius/OBLITERATUS/actions/runs/31872157242.
+
+The operator has now placed ordinary feature work back behind a testing-depth
+gate. Gate 2 remains the verified floor, but it is no longer sufficient by
+itself to start unrelated feature work. The Gate 3 plan below must first be
+adopted and its required work completed. Correctness, security, data-loss, and
+test-infrastructure repairs remain permitted when narrowly scoped and paired
+with reproducing tests.
 
 ## Reasoning
 
@@ -239,7 +255,9 @@ documented critical vertical slices, and no regression in the Gate 1 targets.
 The complete lane must remain below ten minutes per Python version and produce
 JUnit, coverage, package, repeat, mutation, and normalized trend evidence.
 
-Gate 2 permits ordinary new feature work to resume under the per-PR contract.
+Gate 2 originally permitted ordinary new feature work to resume under the
+per-PR contract. The later operator Gate 3 below supersedes that release
+decision.
 
 ### Wave C — property, mutation, determinism, and regression depth
 
@@ -255,6 +273,139 @@ Gate 2 permits ordinary new feature work to resume under the per-PR contract.
 5. Track duration by test and marker, fail on unowned slow-test growth, and
    keep quarantines time-bounded, issue-linked, and absent from critical paths.
 
+### Gate 3 — significant testing depth before new feature work
+
+Gate 3 converts Wave C from an open-ended improvement direction into the next
+mandatory delivery gate. Test count is not a success metric. Each increment
+must protect a named behavior, fail under a deliberate implementation defect,
+and improve at least one of oracle strength, failure-path coverage, mutation
+resistance, environment evidence, or deterministic replay.
+
+#### Quantitative exit criteria
+
+| Measure | Current | Gate 3 minimum |
+|---|---:|---:|
+| Repository statements | 75.80% | 80.00% |
+| Repository branches | 61.89% | 65.00% |
+| Mature CPU statements | 92.89% | 94.00% |
+| Mature CPU branches | 81.72% | 84.00% |
+| Changed executable lines | 90.00% floor | 95.00% floor |
+| Selected mutation score | 91.31% on narrow scope | at least 85% on expanded scope |
+| Unexpected warnings | 0 | 0 |
+| Active flaky quarantines | 0 | 0 |
+| Software conditional evidence age | at most 8 days | at most 8 days |
+
+The expanded mutation scope must include core mutation math, architecture and
+loader decisions, persistence/atomicity, evaluation/report serialization, and
+at least one service-orchestration state machine. Every applicable mutant on
+new or changed pure contract logic must be killed; the aggregate 85% floor is
+not permission to leave changed behavior weakly specified.
+
+The complete mandatory lane must remain under ten minutes per Python version.
+If deeper campaigns exceed that budget, deterministic PR selections remain
+blocking while the complete mutation, repeat, and conditional campaigns run as
+required exact-head or scheduled gates. A failed required campaign may not be
+converted to green by a retry.
+
+#### Risk-ranked work packages
+
+1. **Numerical oracles and research invariants.** Add property and metamorphic
+   tests for projection idempotence, orthogonality, norm bounds, permutation and
+   label invariance, dtype/tolerance behavior, singular and non-finite inputs,
+   deterministic seeds, and serialization stability. Use independent reference
+   calculations for small tensors instead of asserting against the production
+   helper's own intermediate values.
+2. **Mutation and architecture decisions.** Expand selective mutation into
+   refusal-direction selection, architecture traversal, device/dtype and
+   quantization decisions, evaluation routing, and report construction. Add
+   missing negative cases before accepting a surviving mutant as equivalent.
+3. **Failure injection and destructive operations.** Exercise partial writes,
+   fsync/replace failures, corrupt or truncated checkpoints, cancellation at
+   each state transition, cleanup failures, idempotent retry, concurrent writers,
+   and save/reload invariants. No test may risk operator data; use isolated
+   temporary filesystems and fault-injected adapters.
+4. **Orchestration, services, and UI decision seams.** Extract only the minimal
+   clock, transport, process, filesystem, and input/output seams needed to test
+   BESTIARY sync, model catalog resolution, watchtower scheduling, tournament
+   state, interactive flows, and UI construction deterministically. Real
+   networking and UI launch remain in conditional gates.
+5. **Tiny-runtime and quantized semantics.** Extend the pinned tiny-model slice
+   through model mutation, norm restoration, checkpoint round trip, evaluation,
+   and report generation. Add integer/quantized storage semantics, tied/shared
+   weights, unsupported layouts, and explicit numerical-loss expectations before
+   integrating FP8/NVFP4 or large-model offload work.
+6. **Evidence integrity and duration ownership.** Record per-test duration,
+   marker, seed/order, exact candidate SHA, dependency lock, coverage delta,
+   mutation result, and conditional environment. Add policy tests that reject
+   stale, empty, skipped, or wrong-SHA evidence and unowned slow-test growth.
+
+#### Coverage targets for the largest current gaps
+
+These targets apply to CPU-testable decision logic, not to real accelerator,
+model, service, or browser execution. Where a module mixes both, extract a
+small pure seam and measure it rather than mocking away the behavior under test.
+
+| Surface | Current statements / branches | Gate 3 intent |
+|---|---:|---|
+| `abliterate.py` | 45.78% / 37.31% | at least 52% / 45% with mutation/math contracts |
+| `lora_ablation.py` | 7.19% / 0% | at least 70% / 55% for CPU-testable decisions |
+| `tourney.py` | 9.28% / 0% | at least 55% / 40% for lifecycle/state transitions |
+| `bestiary_sync.py` | 0% / 0% | at least 80% / 65% using a loopback/fake transport |
+| `models_client.py` | 19.54% / 3.57% | at least 80% / 65% for resolution/error contracts |
+| `watchtower.py` | 19.19% / 0% | at least 70% / 55% with injected clock/client seams |
+| `interactive.py` | 8.47% / 0% | at least 70% / 55% with scripted I/O contracts |
+| `local_ui.py` | 40.87% / 15.22% | at least 70% / 55% for launch decisions and failures |
+| `ui_watchtower.py` | 0% / 0% | cover construction/callback contracts; real UI stays conditional |
+| `evaluation/heretic_eval.py` | 42.46% / 30.51% | at least 70% / 55% for CPU-testable evaluation logic |
+
+#### Bounded delivery sequence
+
+Each item is an independently reviewable PR and must be exact-head audited
+before merge. Later items may be split further; they may not be combined into a
+single repository-wide coverage PR.
+
+1. Adopt Gate 3 policy, refresh current measurements, and add duration-budget
+   enforcement without changing production behavior.
+2. Add numerical/reference-oracle and metamorphic contracts; expand mutation to
+   the corresponding pure math.
+3. Add loader, architecture, dtype, quantization, and shared-weight decision
+   contracts with mutation evidence.
+4. Add checkpoint failure injection, concurrency, atomicity, and retry contracts.
+5. Add BESTIARY/model-client/watchtower transport and scheduler state contracts.
+6. Add tournament, interactive, and UI decision-seam contracts.
+7. Extend the installed tiny-model vertical slice and quantized-storage
+   semantics, including the exact Float-to-integer restoration regression.
+8. Reconcile conditional software evidence, then run or explicitly waive each
+   unavailable hardware/remote environment under the claim policy.
+9. Publish the Gate 3 execution, coverage, mutation, repeat, duration, and
+   conditional-evidence reports at one canonical `main` commit.
+
+#### Per-PR acceptance evidence
+
+Every Gate 3 PR must publish:
+
+- the exact base and head commits and a clean exact-head audit;
+- named contracts and the defect/mutant each new test detects;
+- focused and mandatory-suite results with warning and skip counts;
+- before/after repository and touched-module line/branch coverage;
+- changed-line coverage and exact-base no-regression results;
+- mutation results when a selected surface changes;
+- runtime delta and slowest affected tests;
+- conditional evidence or an explicit statement that no mapped environment was
+  touched;
+- all required hosted checks green and no unresolved review thread.
+
+#### New-work release decision
+
+Ordinary feature work may resume only after items 1–7 are merged, all Gate 3
+quantitative criteria pass on one canonical post-merge commit, and the final
+reports are reviewed. Item 8 may use a time-bounded waiver only when a required
+runner or credential is unavailable; that waiver blocks support claims for the
+environment and may not weaken CPU gates. Item 9 records the decision and exact
+evidence. Outstanding feature PRs may be audited meanwhile, but only correctness,
+security, data-loss, CI/test-infrastructure fixes, and the Gate 3 testing PRs are
+merge candidates during the pause.
+
 ### Wave D — conditional and release evidence
 
 1. Require fresh successful model-download, external-evaluation, network, and