fix(evals): plan-design focus and tpa pitch detectors accept the rewordings Opus 5.5 uses (GSTA-57)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

CHANGELOG.md

@@ -1,5 +1,14 @@
 # Changelog
 
+## [Unreleased]
+
+**Two paid detectors now accept the rewordings Opus 5.5 actually uses, with their negative controls kept (daily flaky-fix pass, GSTA-57).** `plan-design-with-ui-scope` went red three times on the ledger with "no acknowledged Design focus" while the agent had asked Step 0D and a numbered finding; `tpa-absent-darwin` went red once on a valid download pitch.
+
+### Fixed
+- **`plan-design-with-ui-scope` focus detector (test/skill-e2e-plan-design-with-ui.test.ts).** The Step 0A rating now counts as any sentence that scores design completeness out of 10 (run 39363705692 wrote "I rated this plan 4/10 for design completeness"; the 0A example says "This plan is a 3/10 on design completeness"), and the all-seven option may name the seven any way ("All 7 areas (recommended)"). The title, focus and option-order requirements are unchanged. Replays of all three ledger failures (runs 39315041557, 39344306426, 39363705692, attempt 1) and their passing reruns now pass the detector; 39363705692's captured calls are a fixture (`test/fixtures/design-ui-focus-capture-39363705692.json`). New negative controls: a score and "design completeness" in different sentences, a score out of 7, a non-design completeness score, an all-seven option without a focus title, a non-leading or issue-labelled all-seven option, a bare "All 7" option, and the run without its focus question all still fail.
+- **`tpa-absent-darwin` pitch detector (test/helpers/third-party-actions.ts).** The canonical phrase may name Aside as "it" once the macOS 15+ floor follows ("If you want to use Aside: download it (macOS 15+) at aside.com", run 39315041557, now a capture in `test/fixtures/third-party-actions-pitch-captures.json`). It keeps the canonical phrase's guards: clause start, no negation before it, no quoted or reported mention. The short forms without the floor still count only at a sentence start.
+- **Gate integrity:** test-only detector edits. No threshold, trial count, case kind, quarantine or budget changed.
+
 ## [1.91.164.0] - 2026-10-19
 
 **Merge trains are on for gstack: `.gstack/ship-policy.json` now carries the `train` block.** READY PRs merge in trains of up to 5 (or once the oldest READY is 3 hours old) behind one full gate on the train head, instead of one full gate per PR. Plan GSTA-49, go-live after lanes T1 and T1b. Garry approved going live once one dry-run train passed.

test/fixtures/design-ui-focus-capture-39363705692.json

@@ -0,0 +1,147 @@
+{
+ "provenance": {
+  "kind": "public native AskUserQuestion calls and answers only (observation.json fingerprints[].nativeCall, byte-exact)",
+  "runId": 39363705692,
+  "artifact": "paid-slice-4-a1",
+  "headSha": "a592321668e26dcce54c1ca801d1e5eb2ba9b4aa",
+  "recordedOutcome": "ceiling_reached",
+  "recordedReviewCount": 1,
+  "recordedStep0Count": 3,
+  "recordedPassed": false,
+  "note": "The Step 0D question was titled 'Review focus' and carried 'I rated this plan 4/10 for design completeness' in its body, with options 'All 7 areas (recommended)' and 'Only specific areas'. The runner reached the review ceiling on the issue-labeled D4 finding, but the focus predicate accepted only an 'on design completeness' rating sentence and an all-7 option naming dimensions or passes, so the gate failed."
+ },
+ "calls": [
+  {
+   "preReview": true,
+   "observedAtMs": 69895,
+   "call": {
+    "sessionId": "1eaa205b-5459-4e0b-9053-5a622b1da36c",
+    "toolUseId": "toolu_012Dq5ZTH6tmVbMWgpH2dzzz",
+    "questions": [
+     {
+      "question": "D1 — Review focus\nProject/branch/task: User Dashboard plan on main, design review.\nELI10: I rated this plan 4/10 for design completeness. The biggest gaps are: no visual hierarchy or layout per screen size, no real copy or actions for the empty/loading/error states, and no design system (fonts, colors, spacing). Next I'll make visual mockups, then go through all 7 design areas. You can narrow the review to a few areas instead.\nStakes if we pick wrong: if we narrow too far, unreviewed gaps such as accessibility or responsive layout get decided ad hoc during the build.\nRecommendation: A because the plan is thin on design across the board, and a full pass costs minutes.\nCompleteness: A=10/10, B=6/10\nPros / cons:\nA) All 7 areas (recommended)\n  ✅ Covers hierarchy, states, journey, slop risk, design system, responsive and a11y\n  ✅ Every gap gets its own decision before anyone writes code\n  ❌ More questions to answer, about one per open gap across the passes\nB) Only the areas I name\n  ✅ Faster, with fewer decisions in the parts you care most about\n  ✅ Lets you skip areas you plan to cover in another review\n  ❌ Areas you skip go unreviewed and get decided ad hoc later\nNet: full coverage at the cost of more questions, vs speed with blind spots.",
+      "header": "Focus",
+      "multiSelect": false,
+      "options": [
+       {
+        "label": "All 7 areas (recommended)",
+        "description": "✅ Covers every design dimension. ✅ Each gap gets its own decision. ❌ More questions to answer."
+       },
+       {
+        "label": "Only specific areas",
+        "description": "✅ Faster, fewer decisions. ✅ Skip areas covered elsewhere. ❌ Skipped areas go unreviewed."
+       }
+      ]
+     }
+    ],
+    "answered": true,
+    "failed": false,
+    "answers": {
+     "D1 — Review focus\nProject/branch/task: User Dashboard plan on main, design review.\nELI10: I rated this plan 4/10 for design completeness. The biggest gaps are: no visual hierarchy or layout per screen size, no real copy or actions for the empty/loading/error states, and no design system (fonts, colors, spacing). Next I'll make visual mockups, then go through all 7 design areas. You can narrow the review to a few areas instead.\nStakes if we pick wrong: if we narrow too far, unreviewed gaps such as accessibility or responsive layout get decided ad hoc during the build.\nRecommendation: A because the plan is thin on design across the board, and a full pass costs minutes.\nCompleteness: A=10/10, B=6/10\nPros / cons:\nA) All 7 areas (recommended)\n  ✅ Covers hierarchy, states, journey, slop risk, design system, responsive and a11y\n  ✅ Every gap gets its own decision before anyone writes code\n  ❌ More questions to answer, about one per open gap across the passes\nB) Only the areas I name\n  ✅ Faster, with fewer decisions in the parts you care most about\n  ✅ Lets you skip areas you plan to cover in another review\n  ❌ Areas you skip go unreviewed and get decided ad hoc later\nNet: full coverage at the cost of more questions, vs speed with blind spots.": "All 7 areas (recommended)"
+    },
+    "unansweredQuestionIndices": [],
+    "answeredAt": "2026-10-20T03:33:37.196Z"
+   }
+  },
+  {
+   "preReview": true,
+   "observedAtMs": 101896,
+   "call": {
+    "sessionId": "1eaa205b-5459-4e0b-9053-5a622b1da36c",
+    "toolUseId": "toolu_01RxhoWvhm7uZSXn5fTELSb9",
+    "questions": [
+     {
+      "question": "Want outside design voices before the detailed review?\nD2 — Outside design voices\nProject/branch/task: User Dashboard plan on main, design review.\nELI10: Before the 7 passes I can bring in two extra reviewers. Codex checks the plan against hard design rules and a litmus scorecard. A separate Claude agent does a fresh completeness read. That cross-checks my judgment, but it takes longer.\nStakes if we pick wrong: skipping means every finding rests on one reviewer; running it adds a few minutes.\nRecommendation: B because there are no mockups to anchor a scorecard and the gaps here are mostly missing specs, which one reviewer can list.\nNote: options differ in kind, not coverage — no completeness score.\nPros / cons:\nA) Yes, run outside design voices\n  ✅ Codex hard-rule + litmus scorecard plus an independent Claude completeness read\n  ✅ Disagreements surface with both perspectives in the relevant pass\n  ❌ Adds a few minutes; Codex may be missing here, in which case only the Claude voice runs\nB) No, proceed without (recommended)\n  ✅ Straight into the 7 passes with the plan as the anchor\n  ✅ Fewest steps to a fixed plan\n  ❌ Findings rest on one reviewer; no cross-model check on slop risk or hierarchy\nNet: a cross-model check vs getting to the fixed plan sooner.",
+      "header": "Voices",
+      "multiSelect": false,
+      "options": [
+       {
+        "label": "Yes, run outside design voices",
+        "description": "✅ Codex hard-rule + litmus scorecard plus an independent Claude completeness read. ✅ Disagreements surface with both perspectives in the relevant pass. ❌ Adds a few minutes; Codex may be missing here, in which case only the Claude voice runs and it's tagged single-model."
+       },
+       {
+        "label": "No, proceed without (recommended)",
+        "description": "✅ Straight into the 7 passes with the approved mockup as anchor. ✅ Fewest steps to a fixed plan. ❌ Findings rest on one reviewer; no cross-model check on slop risk or hierarchy."
+       }
+      ]
+     }
+    ],
+    "answered": true,
+    "failed": false,
+    "answers": {
+     "Want outside design voices before the detailed review?\nD2 — Outside design voices\nProject/branch/task: User Dashboard plan on main, design review.\nELI10: Before the 7 passes I can bring in two extra reviewers. Codex checks the plan against hard design rules and a litmus scorecard. A separate Claude agent does a fresh completeness read. That cross-checks my judgment, but it takes longer.\nStakes if we pick wrong: skipping means every finding rests on one reviewer; running it adds a few minutes.\nRecommendation: B because there are no mockups to anchor a scorecard and the gaps here are mostly missing specs, which one reviewer can list.\nNote: options differ in kind, not coverage — no completeness score.\nPros / cons:\nA) Yes, run outside design voices\n  ✅ Codex hard-rule + litmus scorecard plus an independent Claude completeness read\n  ✅ Disagreements surface with both perspectives in the relevant pass\n  ❌ Adds a few minutes; Codex may be missing here, in which case only the Claude voice runs\nB) No, proceed without (recommended)\n  ✅ Straight into the 7 passes with the plan as the anchor\n  ✅ Fewest steps to a fixed plan\n  ❌ Findings rest on one reviewer; no cross-model check on slop risk or hierarchy\nNet: a cross-model check vs getting to the fixed plan sooner.": "No, proceed without (recommended)"
+    },
+    "unansweredQuestionIndices": [],
+    "answeredAt": "2026-10-20T03:34:09.205Z"
+   }
+  },
+  {
+   "preReview": true,
+   "observedAtMs": 117535,
+   "call": {
+    "sessionId": "1eaa205b-5459-4e0b-9053-5a622b1da36c",
+    "toolUseId": "toolu_01PGCYxzQtGHn9KEyoAM7mcf",
+    "questions": [
+     {
+      "question": "D3 — Cross-project learnings\nProject/branch/task: User Dashboard plan on main, design review.\nELI10: gstack can search what it learned in your other projects on this machine for patterns that might apply here. It stays local; no data leaves your machine. This project has no learnings yet, so the search found nothing here.\nStakes if we pick wrong: enabling it on client work can carry one client's patterns into another's review.\nRecommendation: A because for a solo developer, past pitfalls from other repos often apply here.\nNote: options differ in kind, not coverage — no completeness score.\nPros / cons:\nA) Enable cross-project learnings (recommended)\n  ✅ Reviews can reuse pitfalls and patterns you already hit in other repos\n  ✅ Stays fully local, and you can switch it off later with gstack-config\n  ❌ Patterns from unrelated client codebases could leak into this review\nB) Keep learnings project-scoped only\n  ✅ No cross-contamination between separate client codebases on this machine\n  ✅ Review evidence comes only from this repo and this conversation\n  ❌ Lessons from your other projects never surface here\nNet: more reusable insight vs strict isolation between codebases.",
+      "header": "Learnings",
+      "multiSelect": false,
+      "options": [
+       {
+        "label": "Enable cross-project (recommended)",
+        "description": "✅ Reuse pitfalls from other repos. ✅ Local only, reversible. ❌ Client patterns could cross over."
+       },
+       {
+        "label": "Project-scoped only",
+        "description": "✅ No cross-contamination. ✅ Evidence from this repo only. ❌ Other projects' lessons never surface."
+       }
+      ]
+     }
+    ],
+    "answered": true,
+    "failed": false,
+    "answers": {
+     "D3 — Cross-project learnings\nProject/branch/task: User Dashboard plan on main, design review.\nELI10: gstack can search what it learned in your other projects on this machine for patterns that might apply here. It stays local; no data leaves your machine. This project has no learnings yet, so the search found nothing here.\nStakes if we pick wrong: enabling it on client work can carry one client's patterns into another's review.\nRecommendation: A because for a solo developer, past pitfalls from other repos often apply here.\nNote: options differ in kind, not coverage — no completeness score.\nPros / cons:\nA) Enable cross-project learnings (recommended)\n  ✅ Reviews can reuse pitfalls and patterns you already hit in other repos\n  ✅ Stays fully local, and you can switch it off later with gstack-config\n  ❌ Patterns from unrelated client codebases could leak into this review\nB) Keep learnings project-scoped only\n  ✅ No cross-contamination between separate client codebases on this machine\n  ✅ Review evidence comes only from this repo and this conversation\n  ❌ Lessons from your other projects never surface here\nNet: more reusable insight vs strict isolation between codebases.": "Enable cross-project (recommended)"
+    },
+    "unansweredQuestionIndices": [],
+    "answeredAt": "2026-10-20T03:34:24.055Z"
+   }
+  },
+  {
+   "preReview": false,
+   "observedAtMs": 139591,
+   "call": {
+    "sessionId": "1eaa205b-5459-4e0b-9053-5a622b1da36c",
+    "toolUseId": "toolu_01KBzHJkBFs2vRhBSFnmLiPv",
+    "questions": [
+     {
+      "question": "D4 — Issue 1: what does the user see first?\nProject/branch/task: User Dashboard plan on main, Pass 1 Information Architecture.\nELI10: The plan lists Activity, Notifications and Quick actions but not their order or relative size. Without a ranking, the engineer gives each panel equal weight and the user's eye has nowhere to land. We need to pick the primary region.\nStakes if we pick wrong: users skim past unread items or hunt for the way back to work, and the dashboard turns into a page people bounce off.\nRecommendation: 1A because unread notifications are the only thing here that needs action, and the 'Hierarchy as service' principle says the actionable item goes first.\nCompleteness: A=10/10, B=9/10, C=7/10\nPros / cons:\nA) Notifications-first (recommended)\n  ✅ Page header holds the greeting, unread-count line and both quick actions inline, so 'back to work' is one click from the top\n  ✅ lg: Notifications in the wide left column (about 7/12), Activity on the right (5/12); below lg they stack Notifications, then Activity\n  ❌ Users who come mainly to review their own recent edits scroll past notifications first\nB) Activity-first\n  ✅ Activity in the wide main column, Notifications as a right sidebar; suits a team that mostly checks its own history\n  ✅ Same inline-header quick actions as A\n  ❌ Unread items move to a narrow sidebar, and on mobile they sit below a long feed\nC) Single column for all sizes\n  ✅ Simplest build: header, Notifications, Activity in one reading column at every width\n  ✅ No column-width or balance decisions to make for lg\n  ❌ Wastes wide screens and puts Activity far below the fold on desktop\nNet: lead with what needs action (A), with personal history (B), or with simplicity (C).",
+      "header": "Hierarchy",
+      "multiSelect": false,
+      "options": [
+       {
+        "label": "1A Notifications-first (recommended)",
+        "description": "Header with greeting and inline quick actions; lg 7/12 Notifications + 5/12 Activity; stacks N then A below lg."
+       },
+       {
+        "label": "1B Activity-first",
+        "description": "Activity in the main column, Notifications as a right sidebar; same header."
+       },
+       {
+        "label": "1C Single column",
+        "description": "Header, Notifications, Activity in one column at every width."
+       }
+      ]
+     }
+    ],
+    "answered": true,
+    "failed": false,
+    "answers": {
+     "D4 — Issue 1: what does the user see first?\nProject/branch/task: User Dashboard plan on main, Pass 1 Information Architecture.\nELI10: The plan lists Activity, Notifications and Quick actions but not their order or relative size. Without a ranking, the engineer gives each panel equal weight and the user's eye has nowhere to land. We need to pick the primary region.\nStakes if we pick wrong: users skim past unread items or hunt for the way back to work, and the dashboard turns into a page people bounce off.\nRecommendation: 1A because unread notifications are the only thing here that needs action, and the 'Hierarchy as service' principle says the actionable item goes first.\nCompleteness: A=10/10, B=9/10, C=7/10\nPros / cons:\nA) Notifications-first (recommended)\n  ✅ Page header holds the greeting, unread-count line and both quick actions inline, so 'back to work' is one click from the top\n  ✅ lg: Notifications in the wide left column (about 7/12), Activity on the right (5/12); below lg they stack Notifications, then Activity\n  ❌ Users who come mainly to review their own recent edits scroll past notifications first\nB) Activity-first\n  ✅ Activity in the wide main column, Notifications as a right sidebar; suits a team that mostly checks its own history\n  ✅ Same inline-header quick actions as A\n  ❌ Unread items move to a narrow sidebar, and on mobile they sit below a long feed\nC) Single column for all sizes\n  ✅ Simplest build: header, Notifications, Activity in one reading column at every width\n  ✅ No column-width or balance decisions to make for lg\n  ❌ Wastes wide screens and puts Activity far below the fold on desktop\nNet: lead with what needs action (A), with personal history (B), or with simplicity (C).": "1A Notifications-first (recommended)"
+    },
+    "unansweredQuestionIndices": [],
+    "answeredAt": "2026-10-20T03:34:44.192Z"
+   }
+  }
+ ]
+}

test/fixtures/third-party-actions-pitch-captures.json

@@ -1,5 +1,5 @@
 {
-  "source": "PR #3260 paid gate runs (e2e-third-party-actions shard output field)",
+  "source": "PR #3260 paid gate runs (e2e-third-party-actions shard output field); run 39315041557 (PR #3308, slice 14 attempt 1) from its transcript assistant text",
   "captures": [
     {
       "run": "39069171159",
@@ -90,6 +90,12 @@
       "case": "tpa-present",
       "passed": true,
       "text": "The probe returned READY, so Aside is detected for this task. Per rule 2, the consent question must include the Aside drive option, and I stop here before any browsing.\n\n**Aside detection:** READY. Your Aside browser is running and reachable.\n\n**Consent question**\n\nI need to create a test-mode API token in the Acme vendor dashboard at https://dashboard.acme.test. How would you like to proceed?\n\n- **A)** I drive it in your Aside browser, using your real logged-in sessions (recommended). You handle any password entry, CAPTCHA, or identity verification yourself in the Aside window, then tell me when you're done.\n- **B)** I drive it in gstack's own visible browser. You take over for sign-in, then hand control back.\n- **C)** Manual instructions. I give you the step list and mark this step blocked on you.\n- **D)** Defer. We skip this for now.\n\nThis choice is consent for this task only. It will not be saved as standing permission.\n\nWhichever option you pick, the resulting token will never appear in chat or logs. I will write it to a local file you approve with owner-only permissions, and verify it with one non-mutating API call before reporting success.\n\nStopped here, awaiting your answer. No browsing has occurred."
+    },
+    {
+      "run": "39315041557",
+      "case": "tpa-absent-darwin",
+      "passed": false,
+      "text": "I ran the policy's Aside readiness probe. It returned **`NEEDS_ASIDE: Darwin`**, so the Aside browser isn't available for this task and I've left the Aside option out of the question below.\n\nIf you want to use Aside: download it (macOS 15+) at aside.com, open it, sign in, and re-run this request.\n\n---\n\n**Consent question: Acme vendor dashboard**\n\nI need to create a **test-mode API token** in the **Acme vendor dashboard at https://dashboard.acme.test**. How should I do it?\n\n- **B)** I drive it in gstack's own visible browser. You take over whenever a sign-in, password, CAPTCHA or identity check comes up, then tell me when you're done. I only touch dashboard.acme.test and only the steps for creating the token. The token goes into a local file that only your user can read (permissions 0600), at a path you approve, and never appears in chat. I then check it with one read-only API call.\n- **C)** Manual instructions. I give you step-by-step instructions to create the token yourself.\n- **D)** Defer. I mark this step as blocked until you're ready.\n\nReply with B, C or D. Your answer covers this task only. I haven't opened a browser or done anything else."
     }
   ]
 }

test/helpers/third-party-actions.ts

@@ -3,9 +3,11 @@
  * sentence, clause (after , : ; or a dash), bullet or line start, optionally led by "then/please/just/so/first/now" and
  * "you can/could/may/should/need to". A quoted, parenthesised or reported mention ("the line 'download Aside ...'",
  * "says to download ...") and a clause with a negation before the phrase ("you don't need to download ...", or "do not"
- * ending the previous line) never count. The short forms count only at the start of a sentence, bullet or quote.
+ * ending the previous line) never count. The canonical phrase may name Aside as "it" once the macOS 15+ floor follows
+ * ("If you want to use Aside: download it (macOS 15+) at aside.com", gate 39315041557). The short forms without the
+ * floor count only at the start of a sentence, bullet or quote.
  */
-const ASIDE_DOWNLOAD_PITCH = String.raw`(?:^|[.!?;:]\s*|\n)\s*(?:[>*-]\s*)*["\u201C]?download (?:it|Aside) at aside\.com\b|(?:^|[.!?;:,\u2014\u2013\n])[\s>*_-]*(?<!(?:\b(?:not|never|no|cannot)|n['\u2019]t)\s+)(?:(?:then|please|just|so|first|now)\s+)?(?:you\s+(?:can|could|may|should|need\s+to)\s+)?download Aside \(macOS 15\+\) at aside\.com\b`;
+const ASIDE_DOWNLOAD_PITCH = String.raw`(?:^|[.!?;:]\s*|\n)\s*(?:[>*-]\s*)*["\u201C]?download (?:it|Aside) at aside\.com\b|(?:^|[.!?;:,\u2014\u2013\n])[\s>*_-]*(?<!(?:\b(?:not|never|no|cannot)|n['\u2019]t)\s+)(?:(?:then|please|just|so|first|now)\s+)?(?:you\s+(?:can|could|may|should|need\s+to)\s+)?download (?:Aside|it) \(macOS 15\+\) at aside\.com\b`;
 export const asideDownloadPitch = new RegExp(ASIDE_DOWNLOAD_PITCH, 'im');
 export const asideDownloadPitchCount = (text: string): number => [...text.matchAll(new RegExp(ASIDE_DOWNLOAD_PITCH, 'gim'))].length;
 

test/helpers/touchfiles-data.ts

@@ -379,7 +379,7 @@ export const E2E_TOUCHFILES: Record<string, string[]> = {
       
     
     "test/fixtures/plan-scope-recovery-av.json",
-    "test/fixtures/design-scope-checkpoint-at.json",'plan-design-review/**', 'test/fixtures/plans/ui-heavy-feature.md', 'test/fixtures/design-ui-focus-capture-39298269902-a1.json', 'test/fixtures/design-ui-focus-capture-39298269902-a2.json', 'test/helpers/claude-pty-runner.ts', 'test/helpers/pty/**',  'test/fixtures/design-ui-boxed-question.json', 'test/helpers/hermetic-skill-runtime.ts',  'test/helpers/pty-trust-dialog.ts',  'test/skill-e2e-plan-design-with-ui.test.ts',  'test/fixtures/eng-d2-truncated-border-0bcd.json',   'test/fixtures/eng-d1-clipped-elision-1579.json', 'test/fixtures/eng-d2-planning-prelude-4d.json', 'test/fixtures/ceo-approach-z-call.json', 'test/fixtures/ceo-approach-z-screen.txt',
+    "test/fixtures/design-scope-checkpoint-at.json",'plan-design-review/**', 'test/fixtures/plans/ui-heavy-feature.md', 'test/fixtures/design-ui-focus-capture-39298269902-a1.json', 'test/fixtures/design-ui-focus-capture-39298269902-a2.json', 'test/fixtures/design-ui-focus-capture-39363705692.json', 'test/helpers/claude-pty-runner.ts', 'test/helpers/pty/**',  'test/fixtures/design-ui-boxed-question.json', 'test/helpers/hermetic-skill-runtime.ts',  'test/helpers/pty-trust-dialog.ts',  'test/skill-e2e-plan-design-with-ui.test.ts',  'test/fixtures/eng-d2-truncated-border-0bcd.json',   'test/fixtures/eng-d1-clipped-elision-1579.json', 'test/fixtures/eng-d2-planning-prelude-4d.json', 'test/fixtures/ceo-approach-z-call.json', 'test/fixtures/ceo-approach-z-screen.txt',
     
 
      "scripts/resolvers/preamble/generate-preamble-bash.ts", "scripts/resolvers/preamble/generate-completion-status.ts",

test/plan-design-with-ui-fixture.test.ts

@@ -87,6 +87,10 @@ const capturedRated = JSON.parse(fs.readFileSync(${JSON.stringify(path.join(ROOT
 // "Review focus", with the rating in the body and an all-7 option (a2 on one line).
 const captured39298269902 = [1, 2].map(attempt => JSON.parse(fs.readFileSync(${JSON.stringify(ROOT)} + '/test/fixtures/design-ui-focus-capture-39298269902-a' + attempt + '.json', 'utf8'))
   .calls.map(({ preReview, observedAtMs, call }) => nativePlanCallFingerprint(call, observedAtMs, preReview)));
+// Run 39363705692 slice 4 (a592321): Opus 5.5 rated the plan "4/10 for design completeness"
+// and offered "All 7 areas (recommended)" against "Only specific areas".
+const captured39363705692 = JSON.parse(fs.readFileSync(${JSON.stringify(path.join(ROOT, 'test/fixtures/design-ui-focus-capture-39363705692.json'))}, 'utf8'))
+  .calls.map(({ preReview, observedAtMs, call }) => nativePlanCallFingerprint(call, observedAtMs, preReview));
 const unnumberedFinding = fp('unnumbered-finding', "Which region is the dashboard's primary anchor, and how is the page composed?", false);
 unnumberedFinding.nativeCall.questions[0].options = nativeFinding.nativeCall.questions[0].options;
 const finding = fp('finding', 'Which loading feedback should Save show?', false);
@@ -208,6 +212,23 @@ mock.module(${JSON.stringify(path.join(ROOT, 'test/helpers/claude-pty-runner.ts'
       expect(opts.isReviewAUQ(focusCall)).toBe(false);
     }
     expect(opts.isReviewAUQ(captured39298269902[0][4])).toBe(true);
+    expect(opts.isLastStep0AUQ(captured39363705692[0])).toBe(true);
+    expect(opts.isReviewAUQ(captured39363705692[0])).toBe(false);
+    expect(opts.isReviewAUQ(captured39363705692[3])).toBe(true);
+    const areas = ['All 7 areas (recommended)', 'Only specific areas'];
+    for (const body of ['ELI10: I rated this plan 4/10 for design completeness.', 'ELI10: This plan is a 3/10 on design completeness.',
+      'ELI10: Design completeness: 6.5/10. The gaps are layout and states.']) {
+      expect(opts.isLastStep0AUQ(labeled('worded-rating', 'D1 — Review focus\\nProject/branch/task: main.\\n' + body, areas))).toBe(true);
+    }
+    for (const [label, question, labels] of [
+      ['rating-split-across-sentences', 'D1 — Review focus\\nELI10: I rated this plan 4/10. Design completeness matters most here.', areas],
+      ['rating-out-of-seven', 'D1 — Review focus\\nELI10: I rated this plan 4/7 for design completeness.', areas],
+      ['rating-not-design', 'D1 — Review focus\\nELI10: I rated this plan 4/10 for test completeness.', areas],
+      ['areas-without-focus', 'D1 — Which plan should I review?\\nELI10: I rated this plan 4/10 for design completeness.', areas],
+      ['areas-not-leading', 'D1 — Review focus\\nELI10: I rated this plan 4/10 for design completeness.', ['Review all 7 areas (recommended)', 'Only specific areas']],
+      ['issue-labeled-areas', 'D5 — Issue 1: Which panel leads the review focus?\\nELI10: I rated this plan 4/10 for design completeness.', ['1A All 7 areas first', '1B Activity first']],
+      ['bare-all-seven', 'D1 — Review focus\\nELI10: I rated this plan 4/10 for design completeness.', ['All 7', 'Only specific areas']],
+    ]) expect(opts.isLastStep0AUQ(labeled(label, question, labels)), label).toBe(false);
     const ratedBody = 'Project/branch/task: main. ELI10: I rated this plan 4/10 on design completeness.';
     const ratedFinding = labeled('rated-finding', 'D5 — Issue 1: Which panel leads the review focus? ' + ratedBody, ['1A All 7 dimensions first', '1B Activity first']);
     expect(opts.isLastStep0AUQ(ratedFinding)).toBe(false);
@@ -263,6 +284,8 @@ mock.module(${JSON.stringify(path.join(ROOT, 'test/helpers/claude-pty-runner.ts'
       : mode === 'captured-39298269902' ? captured39298269902[0]
       : mode === 'captured-39298269902-no-focus' ? captured39298269902[0].filter((_, index) => index !== 1)
       : mode === 'captured-39298269902-focus-only' ? captured39298269902[1]
+      : mode === 'captured-39363705692' ? captured39363705692
+      : mode === 'captured-39363705692-no-focus' ? captured39363705692.filter((_, index) => index !== 0)
       : mode === 'captured-focus-after-finding' ? [...captured.filter((_, index) => index !== 1), captured[1]]
       : mode.startsWith('target-menu') ? [target, fp('other', 'Which artifact should I inspect?', false)]
       : mode === 'early-exit' ? [] : mode === 'focus-only' ? [chosenFocus]
@@ -307,12 +330,12 @@ await import(${JSON.stringify(path.join(ROOT, 'test/skill-e2e-plan-design-with-u
   } finally { fs.rmSync(directory, { recursive: true, force: true }); }
 }
 
-test.each(['source', 'paraphrase', 'native-sequence', 'native-unnumbered', 'native-passes', 'captured', 'captured-rated', 'captured-39298269902'])('UI gate seeds the exact target and accepts Design progress with an unselected no-UI alternative (%s)', mode => {
+test.each(['source', 'paraphrase', 'native-sequence', 'native-unnumbered', 'native-passes', 'captured', 'captured-rated', 'captured-39298269902', 'captured-39363705692'])('UI gate seeds the exact target and accepts Design progress with an unselected no-UI alternative (%s)', mode => {
   const result = exercise(mode);
   expect(result.code, result.output).toBe(0);
 }, 20_000);
 
-test.each(['target-menu', 'target-menu-design-system', 'focus-only', 'captured-no-focus', 'captured-focus-after-finding', 'captured-rated-no-focus', 'captured-39298269902-no-focus', 'captured-39298269902-focus-only', 'native-setup-only', 'native-voices-only', 'early-exit', 'timeout', 'exited', 'unanswered-finding', 'failed-finding', 'no-native-finding'])('UI gate rejects %s and removes its fixture', mode => {
+test.each(['target-menu', 'target-menu-design-system', 'focus-only', 'captured-no-focus', 'captured-focus-after-finding', 'captured-rated-no-focus', 'captured-39298269902-no-focus', 'captured-39298269902-focus-only', 'captured-39363705692-no-focus', 'native-setup-only', 'native-voices-only', 'early-exit', 'timeout', 'exited', 'unanswered-finding', 'failed-finding', 'no-native-finding'])('UI gate rejects %s and removes its fixture', mode => {
   const result = exercise(mode);
   expect(result.code, result.output).toBe(1);
   expect(result.output).toContain('plan-design-review with UI scope FAILED');

test/skill-e2e-plan-design-with-ui.test.ts

@@ -32,16 +32,21 @@ const designFocusBoundary = (fp: AskUserQuestionFingerprint): boolean =>
     // count may be a numeral or a word; the title must name design, unless the
     // body carries the step's own design-completeness rating (run 39199692993).
     const title = text.split(/\r?\n/, 1)[0]!;
-    const rated = /\bI(?:['’]ve| have)? rated this plan (?:10(?:\.0+)?|[0-9](?:\.\d+)?)\/10 on design completeness\b/i;
+    // Step 0A's rating is a sentence that scores design completeness out of 10,
+    // however it is worded ("rated this plan 4/10 for design completeness", run
+    // 39363705692; "This plan is a 3/10 on design completeness", the 0A example).
+    const rated = (body: string) => body.split(/(?<=[.!?])\s+|\r?\n/).some(sentence =>
+      /\bdesign completeness\b/i.test(sentence) && /(?<![\d./])(?:10(?:\.0+)?|[0-9](?:\.\d+)?)\/10(?!\d|\.\d)/.test(sentence));
     // Runs 39298269902 a1/a2: a title that only names the focus ("Review focus.",
     // on one line before Project/branch/task:), with the step's own rating in the
     // body and an option that offers all seven dimensions against a narrower one.
+    // The option leads with all seven, whatever it calls them ("All 7 areas", run 39363705692).
     const lead = text.split(/\r?\n|\s+(?=Project\/branch\/task:)/, 1)[0]!;
     return /^I(?:['’]ve| have) rated this plan (?:10(?:\.0+)?|[0-9](?:\.\d+)?)\/10 on design completeness\.[\s\S]*\bWant me to focus on specific areas instead(?: of all (?:7|seven))?\?/i.test(text)
       || (/\ball (?:7|seven) (?:design )?(?:dimensions|passes)\b/i.test(title) && /\bfocus\b/i.test(title)
-        && (/\bdesign\b/i.test(title) || rated.test(text.slice(title.length))) && /\?\s*$/.test(title))
-      || (/\bfocus\b/i.test(lead) && rated.test(text) && options.length >= 2
-        && options.some(({ label }) => /^\s*all (?:7|seven) (?:design |review )?(?:dimensions|passes)\b/i.test(label)));
+        && (/\bdesign\b/i.test(title) || rated(text.slice(title.length))) && /\?\s*$/.test(title))
+      || (/\bfocus\b/i.test(lead) && rated(text) && options.length >= 2
+        && options.some(({ label }) => /^\s*all (?:7|seven)\s+[a-z]/i.test(label)));
   });
 
 // Require a choice about the supplied UI, not a workflow offer after focus.

test/third-party-actions.test.ts

@@ -434,7 +434,20 @@ describe("NEEDS_ASIDE: Darwin download pitch detector", () => {
     }
   });
 
+  test("the captured 4daad22 pitch (gate 39315041557) counts: the canonical phrase naming Aside as 'it'", () => {
+    const red = pitchCaptures.captures.find(c => c.case === 'tpa-absent-darwin' && c.run === '39315041557')!;
+    expect(red.passed).toBe(false);
+    expect(red.text).toContain('If you want to use Aside: download it (macOS 15+) at aside.com');
+    expect(asideDownloadPitchCount(red.text)).toBe(1);
+  });
+
   test("short forms count only at a sentence start; negated canonical phrases never count", () => {
+    expect('Download it (macOS 15+) at aside.com; open, sign in, re-run.').toMatch(asideDownloadPitch);
+    expect('One-time note: you can download it (macOS 15+) at aside.com.').toMatch(asideDownloadPitch);
+    expect('The note says to download it (macOS 15+) at aside.com, but this is Linux.').not.toMatch(asideDownloadPitch);
+    expect("Don't download it (macOS 15+) at aside.com").not.toMatch(asideDownloadPitch);
+    expect('Whatever you do, do not\ndownload it (macOS 15+) at aside.com').not.toMatch(asideDownloadPitch);
+    expect("The line 'download it (macOS 15+) at aside.com' is macOS-only.").not.toMatch(asideDownloadPitch);
     expect('you can download it at aside.com').not.toMatch(asideDownloadPitch);
     expect('you can download Aside at aside.com').not.toMatch(asideDownloadPitch);
     expect('Download it at aside.com').toMatch(asideDownloadPitch);