garrytan
a111225e78
fix(evals): repair proof-run reds in design-consultation, document-release, design and QA fixtures
...
- design-consultation Phase 1 asks one brief that confirms context and decides
research; the confirm-only first question scored substance 2.
- document-release defines ship-owned inputs, exact steps and the JSON result,
and drops stale spawned-from-/ship text (judge actionability 3.67 -> 4/4/4).
- plan-design-with-ui accepts the Step 0D focus menu the same way the shared
picker does ("focus on specific ones?").
- plan-design-review plan-mode saves in three Edits instead of one final Write.
- QA functional annotations ask for the full 40-character revision.
- Outside-disabled attribution judges quoted prior-record data by its exact
timestamp or a dated, pre-existing-record sentence; four captured phrasings
replay clean and current claims still fail.
- --case can select autoplan-dual-voice by its literal test name.
2026-09-29 22:36:28 +00:00
Garry Tan
dcaea52800
v1.91.7.0 feat: add functional QA and pre-publication docs checks ( #2983 )
...
* feat: add surface-aware exploratory QA and ship documentation gates
* test: preserve delegated QA setup authority after main integration
* fix(qa): clarify exploration order and preserve report artifacts
* test(qa): follow the shared setup reference directly
* refactor(ship): make verification and recovery routes explicit
* test(ship): align evidence and review guards with explicit routes
* fix(workflows): clarify ship recovery and functional QA evidence
* fix(workflows): clarify approval recovery and full QA coverage
* refactor(workflows): order review transactions and clarify ship state
* fix(ship): clarify final verification and fail closed at publication
* fix(evals): attribute native atomic documentation writes
* fix(ship): clarify recovery and documentation lifecycle guidance
* fix(test): preserve observed native placeholder styling in CI
* fix(codex): report watchdog timeouts without a process-exit race
* Checkpoint functional QA implementation and workflow validation repairs
* Fix documentation and shared-review fixture contracts
* docs: clarify judge reuse and evaluation supervision
* test: align review evidence and selected case contracts
* test: verify append-only documentation checkpoints and recovery
* fix: qualify QA workflows and CI validation repairs
* fix: launch shared-libs fixture scripts on Windows
* fix: qualify QA deadlines, fixture isolation, and shard cleanup
* fix: preserve qualified QA and cancellation repairs
* fix: enforce functional fixture authority and share strict event decoding
* fix: retain free-test evidence and explain recovery
* fix: reject malformed native evidence after decoder consolidation
* test: use reliable capture for telemetry privacy filters
* test: refresh measured quick coverage and document validation costs
* Fix native fixture receipts and preserve VM validation evidence
* Align negative judge controls with upstream clarity policy
* Fix report-only QA preparation and public evidence handling
* Clarify QA-only preparation and current-report preservation
* Stream Ship quality judgments with an explicit 64k response contract
* Validate compact judge reasoning locally with supported wire schema
* Align functional QA fixture instructions with evidence acceptance
* Bind native browser diagnostics to execution evidence and align review verdicts
* Preserve native diagnostic line boundaries
* Serialize functional QA evidence from native captures
* Keep large QA evidence fixture payload out of Windows argv
2026-09-29 06:07:35 -07:00
garrytan