fix(plan-reviews): restore RECOMMENDATION + Completeness split + Codex ELI10 (v1.6.3.0) (#1149)

* test: add AskUserQuestion format regression eval for plan reviews

Four-case periodic-tier eval that captures the verbatim AskUserQuestion
text /plan-ceo-review and /plan-eng-review produce, then asserts the
format rule is honored: RECOMMENDATION always, Completeness: N/10 only
on coverage-differentiated options, and an explicit "options differ in
kind" note on kind-differentiated options.

Cases:
- plan-ceo-review mode selection (kind-differentiated)
- plan-ceo-review approach menu (coverage-differentiated)
- plan-eng-review per-issue coverage decision
- plan-eng-review per-issue architectural choice (kind-differentiated)

Classified periodic because behavior depends on Opus non-determinism —
gate-tier would flake and block merges.

Test harness instructs the agent to write its would-be AskUserQuestion
text to $OUT_FILE rather than invoke a real tool (MCP AskUserQuestion
isn't wired in the test subprocess). Regex predicates then validate
the captured content.

Cost: ~$2 per full run.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(plan-reviews): restore RECOMMENDATION + split Completeness by question type

Opus 4.7 users reported /plan-ceo-review and /plan-eng-review stopped
emitting the RECOMMENDATION line and per-option Completeness: X/10
scores. E2E capture showed the real failure mode: on kind-differentiated
questions (mode selection, architectural A-vs-B, cherry-pick), Opus 4.7
either fabricated filler scores (10/10 on every option — conveys nothing)
or dropped the format entirely when the metric didn't fit.

Fix is at two layers:

1. scripts/resolvers/preamble/generate-ask-user-format.ts splits the old
   run-on step 3 into:
   - Step 3 "Recommend (ALWAYS)": RECOMMENDATION is required on every
     question, coverage- or kind-differentiated.
   - Step 4 "Score completeness (when meaningful)": emit Completeness: N/10
     only when options differ in coverage. When options differ in kind,
     skip the score and include a one-line explanatory note. Do not
     fabricate scores.

2. scripts/resolvers/preamble/generate-completeness-section.ts updates
   the Completeness Principle tail to match. Without this, the preamble
   contained two rules (one conditional, one unconditional) and the
   model hedged.

Template anchors reinforce the distinction where agent judgment is most
likely to drift:

- plan-ceo-review Section 0C-bis (approach menu) gets the
  coverage-differentiated anchor.
- plan-ceo-review Section 0F (mode selection) gets the kind-differentiated
  anchor.
- plan-eng-review CRITICAL RULE section gets the coverage-vs-kind rule
  for every per-issue AskUserQuestion raised during the review.

Regenerated SKILL.md for all T2 skills + golden fixtures refreshed. Every
skill using the T2 preamble now has the same conditional scoring rule.

Verified via new periodic-tier eval (test/skill-e2e-plan-format.test.ts):
all 4 cases fail on prior behavior, all 4 pass with this fix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v1.6.2.0)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test: add Codex eval for AskUserQuestion format compliance

Four-case periodic-tier eval mirrors test/skill-e2e-plan-format.test.ts
but drives the plan review skills via codex exec instead of claude -p.

Context: Codex under the gpt.md "No preamble / Prefer doing over listing"
overlay tends to skip the Simplify/ELI10 paragraph and the RECOMMENDATION
line on AskUserQuestion calls. Users have to manually re-prompt "ELI10
and don't forget to recommend" almost every time. This test pins the
behavior so regressions surface.

Cases:
- plan-ceo-review mode selection (kind-differentiated)
- plan-ceo-review approach menu (coverage-differentiated)
- plan-eng-review per-issue coverage decision
- plan-eng-review per-issue architectural choice (kind-differentiated)

Assertions on captured AskUserQuestion text:
- RECOMMENDATION: Choose present (all cases)
- Completeness: N/10 present on coverage, absent on kind
- "options differ in kind" note present on kind
- ELI10 length floor (>400 chars) — catches bare options-only output

Cost: ~\$2-4 per full run.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(preamble): harden AskUserQuestion Format + Codex ELI10 carve-out

Follow-up to v1.6.2.0. Codex (GPT-5.4) under the gpt.md overlay
treated "No preamble / Prefer doing over listing" as license to skip
the Simplify paragraph and the RECOMMENDATION line on AskUserQuestion
calls. Users had to manually re-prompt "ELI10 and don't forget to
recommend" almost every time.

Two layers:

1. model-overlays/gpt.md — adds an explicit "AskUserQuestion is NOT
   preamble" carve-out. The "No preamble" rule applies to direct
   answers; AskUserQuestion content must emit the full format
   (Re-ground, Simplify/ELI10, Recommend, Options). Tells the model:
   if you find yourself about to skip any of these, back up and emit
   them — the user will ask anyway, so do it the first time.

2. scripts/resolvers/preamble/generate-ask-user-format.ts — step 2
   renamed to "Simplify (ELI10, ALWAYS)" with explicit "not optional
   verbosity, not preamble" framing. Step 3 "Recommend (ALWAYS)"
   hardened: "Never omit, never collapse into the options list."

All T2 skills regenerated across all hosts. Golden fixtures refreshed
(claude-ship, codex-ship, factory-ship). Updated the ELI10 assertion
in test/gen-skill-docs.test.ts to match the new wording.

Codex compliance to be verified empirically via test/codex-e2e-plan-format.test.ts.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test: fix Codex eval sandbox + collector API

Two test infrastructure bugs in the initial Codex eval landed in the
prior commit:

1. sandbox: 'read-only' (the default) blocked Codex from writing
   $OUT_FILE. Test reported "STATUS: BLOCKED" and exited 0 without
   a capture file. Fixed: sandbox: 'workspace-write' for all 4 cases,
   allowing writes inside the tempdir.

2. recordCodexResult called a non-existent evalCollector.record()
   API (I invented it). The real surface is addTest() with a
   different field schema. Aligned with test/codex-e2e.test.ts
   pattern.

With both fixed, the eval now actually measures Codex AskUserQuestion
format compliance. All 4 cases pass on v1.6.2.0 with the gpt.md
carve-out: RECOMMENDATION always, Completeness: N/10 only on coverage,
"options differ in kind" note on kind, ELI10 explanation present.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore: bump version and changelog (v1.6.3.0)

Adds the Codex ELI10 + RECOMMENDATION carve-out scope landed after
v1.6.2.0's Claude-verified fix.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

pair-agent/SKILL.md

@@ -465,11 +465,13 @@ available]. [Health score if available]." Keep it to 2-3 sentences.
 
 ## AskUserQuestion Format
 
-**ALWAYS follow this structure for every AskUserQuestion call:**
+**ALWAYS follow this structure for every AskUserQuestion call. All four elements are non-skippable. If you find yourself about to skip any of them, stop and back up.**
+
 1. **Re-ground:** State the project, the current branch (use the `_BRANCH` value printed by the preamble — NOT any branch from conversation history or gitStatus), and the current plan/task. (1-2 sentences)
-2. **Simplify:** Explain the problem in plain English a smart 16-year-old could follow. No raw function names, no internal jargon, no implementation details. Use concrete examples and analogies. Say what it DOES, not what it's called.
-3. **Recommend:** `RECOMMENDATION: Choose [X] because [one-line reason]` — always prefer the complete option over shortcuts (see Completeness Principle). Include `Completeness: X/10` for each option. Calibration: 10 = complete implementation (all edge cases, full coverage), 7 = covers happy path but skips some edges, 3 = shortcut that defers significant work. If both options are 8+, pick the higher; if one is ≤5, flag it.
-4. **Options:** Lettered options: `A) ... B) ... C) ...` — when an option involves effort, show both scales: `(human: ~X / CC: ~Y)`
+2. **Simplify (ELI10, ALWAYS):** Explain what's happening in plain English a smart 16-year-old could follow. Concrete examples and analogies, not function names or internal jargon. Say what it DOES, not what it's called. State the stakes: what breaks if we pick wrong. This is NOT optional verbosity and it is NOT preamble — the user is about to make a decision and needs context. Even if you'd normally stay terse, emit the ELI10 paragraph. The user will ask for it anyway; do it the first time.
+3. **Recommend (ALWAYS):** Every question ends with `RECOMMENDATION: Choose [X] because [one-line reason]` on its own line. Never omit it. Never collapse it into the options list. Required for every AskUserQuestion, regardless of whether the options are coverage-differentiated or different-in-kind.
+4. **Score completeness (when meaningful):** When options differ in coverage (e.g. full test coverage vs happy path vs shortcut, complete error handling vs partial), score each with `Completeness: N/10` on its own line. Calibration: 10 = complete (all edge cases, full coverage), 7 = happy path only, 3 = shortcut. Flag any option ≤5 where a higher-completeness option exists. When options differ in kind (picking a review posture, picking an architectural approach, cherry-pick Add/Defer/Skip, choosing between two different kinds of systems), the completeness axis doesn't apply — skip `Completeness: N/10` entirely and write one line: `Note: options differ in kind, not coverage — no completeness score.` Do not fabricate filler scores.
+5. **Options:** Lettered options: `A) ... B) ... C) ...` — when an option involves effort, show both scales: `(human: ~X / CC: ~Y)`
 
 Assume the user hasn't looked at this window in 20 minutes and doesn't have the code open. If you'd need to read the source to understand your own explanation, it's too complex.
 
@@ -589,7 +591,7 @@ AI makes completeness near-free. Always recommend the complete option over short
 | Feature | 1 week | 30 min | ~30x |
 | Bug fix | 4 hours | 15 min | ~20x |
 
-Include `Completeness: X/10` for each option (10=all edge cases, 7=happy path, 3=shortcut).
+When options differ in coverage (e.g. full vs happy-path vs shortcut), include `Completeness: X/10` on each option (10 = all edge cases, 7 = happy path, 3 = shortcut). When options differ in kind (mode posture, architectural choice, cherry-pick A/B/C where each is a different kind of thing, not a more-or-less-complete version of the same thing), skip the score and write one line explaining why: `Note: options differ in kind, not coverage — no completeness score.` Do not fabricate scores.
 
 ## Confusion Protocol