garrytan
b1f5bc0032
test(evals): retire every paid automatic retry
...
Paid evals never retry (approved 2026-09-29): delete SHORT_CASE_RETRY_FILES
and retriesWithinCaseCap, drop the retry fields from the registered wall rows
(walls now cover one run plus reserve), make retriesForFiles return 0, pass
--retry 0 explicitly, and drop --retry 1 from the package.json paid scripts.
Add the eval:pass-rates alias. Tests that pinned the old retry allowance are
updated as a policy change; review-finalization-budget now proves late-result
recording under the production zero-retry arguments.
2026-09-29 19:09:51 +00:00
Garry Tan
dcaea52800
v1.91.7.0 feat: add functional QA and pre-publication docs checks ( #2983 )
...
* feat: add surface-aware exploratory QA and ship documentation gates
* test: preserve delegated QA setup authority after main integration
* fix(qa): clarify exploration order and preserve report artifacts
* test(qa): follow the shared setup reference directly
* refactor(ship): make verification and recovery routes explicit
* test(ship): align evidence and review guards with explicit routes
* fix(workflows): clarify ship recovery and functional QA evidence
* fix(workflows): clarify approval recovery and full QA coverage
* refactor(workflows): order review transactions and clarify ship state
* fix(ship): clarify final verification and fail closed at publication
* fix(evals): attribute native atomic documentation writes
* fix(ship): clarify recovery and documentation lifecycle guidance
* fix(test): preserve observed native placeholder styling in CI
* fix(codex): report watchdog timeouts without a process-exit race
* Checkpoint functional QA implementation and workflow validation repairs
* Fix documentation and shared-review fixture contracts
* docs: clarify judge reuse and evaluation supervision
* test: align review evidence and selected case contracts
* test: verify append-only documentation checkpoints and recovery
* fix: qualify QA workflows and CI validation repairs
* fix: launch shared-libs fixture scripts on Windows
* fix: qualify QA deadlines, fixture isolation, and shard cleanup
* fix: preserve qualified QA and cancellation repairs
* fix: enforce functional fixture authority and share strict event decoding
* fix: retain free-test evidence and explain recovery
* fix: reject malformed native evidence after decoder consolidation
* test: use reliable capture for telemetry privacy filters
* test: refresh measured quick coverage and document validation costs
* Fix native fixture receipts and preserve VM validation evidence
* Align negative judge controls with upstream clarity policy
* Fix report-only QA preparation and public evidence handling
* Clarify QA-only preparation and current-report preservation
* Stream Ship quality judgments with an explicit 64k response contract
* Validate compact judge reasoning locally with supported wire schema
* Align functional QA fixture instructions with evidence acceptance
* Bind native browser diagnostics to execution evidence and align review verdicts
* Preserve native diagnostic line boundaries
* Serialize functional QA evidence from native captures
* Keep large QA evidence fixture payload out of Windows argv
2026-09-29 06:07:35 -07:00
garrytan