fix(evals): repair proof-run reds in design-consultation, document-release, design and QA fixtures

- design-consultation Phase 1 asks one brief that confirms context and decides
  research; the confirm-only first question scored substance 2.
- document-release defines ship-owned inputs, exact steps and the JSON result,
  and drops stale spawned-from-/ship text (judge actionability 3.67 -> 4/4/4).
- plan-design-with-ui accepts the Step 0D focus menu the same way the shared
  picker does ("focus on specific ones?").
- plan-design-review plan-mode saves in three Edits instead of one final Write.
- QA functional annotations ask for the full 40-character revision.
- Outside-disabled attribution judges quoted prior-record data by its exact
  timestamp or a dated, pre-existing-record sentence; four captured phrasings
  replay clean and current claims still fail.
- --case can select autoplan-dual-voice by its literal test name.
This commit is contained in:
garrytan committed 2026-09-29 22:36:28 +00:00
1 parent aba80c8fb6
commit a111225e78
17 files changed
+180 -92

No files matched your search

+5 -8
View File
@@ -644,15 +644,12 @@ sections. Read a section in full before doing its step; do not work from memory.
## Phase 1: Product Context
Confirm product context in Q1, pre-filled from the codebase; then ask the memorable-thing question.
**AskUserQuestion Q1 — one brief that confirms context AND decides research.** Never ask a confirm-only question first. In the ELI10, state your pre-filled read (from README, product files or office-hours output): what the product is, who it's for, its space and project type (web app, dashboard, marketing site, editorial, internal tool, etc.). Options:
- A) Context right — research what top products in this space do for design first
- B) Context right — work from design knowledge only
- C) Context wrong or incomplete — I'll correct it
**AskUserQuestion Q1 — include ALL of these:**
1. Confirm what the product is, who it's for, what space/industry
2. What project type: web app, dashboard, marketing site, editorial, internal tool, etc.
3. "Want me to research what top products in your space are doing for design, or should I work from my design knowledge?"
4. **Explicitly say:** "At any point you can just drop into chat and we'll talk through anything — this isn't a rigid form, it's a conversation."
Pre-fill context from README or office-hours output, then confirm it and the research preference in Q1.
Recommend A or B for this product, naming what research buys or costs here versus the other. **Explicitly say:** "At any point you can just drop into chat and we'll talk through anything — this isn't a rigid form, it's a conversation."
**Memorable-thing forcing question.** Before moving on, ask the user: *"What's the one
thing you want someone to remember after they see this product for the first time?"*
+5 -8
View File
@@ -125,15 +125,12 @@ Phase 5: `DESIGN_READY` uses AI mockups on realistic product screens; `DESIGN_NO
## Phase 1: Product Context
Confirm product context in Q1, pre-filled from the codebase; then ask the memorable-thing question.
**AskUserQuestion Q1 — one brief that confirms context AND decides research.** Never ask a confirm-only question first. In the ELI10, state your pre-filled read (from README, product files or office-hours output): what the product is, who it's for, its space and project type (web app, dashboard, marketing site, editorial, internal tool, etc.). Options:
- A) Context right — research what top products in this space do for design first
- B) Context right — work from design knowledge only
- C) Context wrong or incomplete — I'll correct it
**AskUserQuestion Q1 — include ALL of these:**
1. Confirm what the product is, who it's for, what space/industry
2. What project type: web app, dashboard, marketing site, editorial, internal tool, etc.
3. "Want me to research what top products in your space are doing for design, or should I work from my design knowledge?"
4. **Explicitly say:** "At any point you can just drop into chat and we'll talk through anything — this isn't a rigid form, it's a conversation."
Pre-fill context from README or office-hours output, then confirm it and the research preference in Q1.
Recommend A or B for this product, naming what research buys or costs here versus the other. **Explicitly say:** "At any point you can just drop into chat and we'll talk through anything — this isn't a rigid form, it's a conversation."
**Memorable-thing forcing question.** Before moving on, ask the user: *"What's the one
thing you want someone to remember after they see this product for the first time?"*