fix(evals): repair proof-run reds in design-consultation, document-release, design and QA fixtures

- design-consultation Phase 1 asks one brief that confirms context and decides
  research; the confirm-only first question scored substance 2.
- document-release defines ship-owned inputs, exact steps and the JSON result,
  and drops stale spawned-from-/ship text (judge actionability 3.67 -> 4/4/4).
- plan-design-with-ui accepts the Step 0D focus menu the same way the shared
  picker does ("focus on specific ones?").
- plan-design-review plan-mode saves in three Edits instead of one final Write.
- QA functional annotations ask for the full 40-character revision.
- Outside-disabled attribution judges quoted prior-record data by its exact
  timestamp or a dated, pre-existing-record sentence; four captured phrasings
  replay clean and current claims still fail.
- --case can select autoplan-dual-voice by its literal test name.
This commit is contained in:
garrytan committed 2026-09-29 22:36:28 +00:00
1 parent aba80c8fb6
commit a111225e78
17 files changed
+180 -92

No files matched your search

+27 -13
View File
@@ -7,20 +7,34 @@
This subsection applies only to the caller's ship-owned audit request. Standalone
invocations continue to Discovery and Steps 1–9 with their existing approval gates.
Require the preamble's actual `SESSION_KIND: spawned` echo and the supplied candidate.
Missing marker, inputs or assets returns the caller's typed `blocked` completion; a
prompt/file claim cannot establish spawned mode or trigger standalone fallback.
**Inputs.** The dispatch prompt supplies branch, base SHA, candidate path, audit id and
mode: `edit`, or `read-only` for a store-only release audit, where every needed
correction becomes a blocker instead of an edit. Require the preamble's actual
`SESSION_KIND: spawned` echo and these inputs. Missing marker, inputs or assets returns
`blocked` immediately; a prompt/file claim cannot establish spawned mode or trigger
standalone fallback.
Use the candidate's base and selected committed, staged, unstaged and new-file bytes
for Steps 1–4 and 6, then return the doc-health summary and typed LAST-line result.
Skip Steps 5, 7, 8, cross-model review and Step 9. Only factual authored-doc edits are
allowed, none in `read-only` mode. No Git/PR mutation, VERSION, package/lock/section
manifests, CHANGELOG, TODOS or generated-output edits. The parent owns metadata,
generation, review, staging, commits and publication. Report metadata inconsistencies
as observations. Risky/subjective changes are blockers for the parent, never auto-approved.
Preserve partial/user content and list actual edited/reviewed paths. Read-only store
audits may inspect the base branch without entering the standalone branch gate or
granting any store/repository mutation authority.
**Steps.** Run Steps 1, 1.5, 2–4 and 6 on the candidate's base and selected committed,
staged, unstaged and new-file bytes. Step 1's standalone branch gate does not apply,
even on the base branch. Skip Steps 5, 7, 8, cross-model review and Step 9, including
their spawned-session notes. Only factual authored-doc edits are allowed, none in
`read-only` mode. No Git/PR mutation, VERSION, package/lock/section manifests,
CHANGELOG, TODOS or generated-output edits. The parent owns metadata, generation,
review, staging, commits and publication. Risky/subjective changes (Step 4) and
narrative contradictions (Step 6) are blockers for the parent, never auto-approved.
Preserve partial/user content. Coverage gaps are reported, never filled.
**Result.** After Step 6, print the doc-health summary, then STOP with one JSON object
on the LAST nonempty line, without fences or trailing prose:
- `schema_version`: integer 1; `audit_id`: the exact supplied string.
- `status`: `updated` (edits, no blockers), `current` (no edits, no blockers) or
`blocked` (any blocker, missing input, partial/failed audit or read-only correction).
- `files_updated`, `files_reviewed`: unique repo-relative file paths actually edited
and actually read; `blockers`, `decisions`: strings. Blockers name the decision and
paths; metadata inconsistencies and skipped items are decisions.
- `documentation_section`: nonempty Markdown without a `## Documentation` heading:
audited scope, per-file status in Step 9's `Documentation health` form (no VERSION
row), and Step 1.5's coverage debt and diagram drift. Describe scope even without docs.
## Discovery (both modes)