fix(plan-tune): map the clicked AskUserQuestion label to the registered option key at capture (#2623) (#3394)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

CHANGELOG.md

@@ -1,5 +1,19 @@
 # Changelog
 
+## [1.91.223.0] - 2026-10-25
+
+**/plan-tune's inferred profile finally moves: a clicked answer is logged as the question's registered option key, not its label.** #2623 showed every inferred dimension stuck at 0.5 on a 1,286-event log: the AskUserQuestion hook recorded labels like "A) Fix it now (recommended)", while the signal map that turns answers into dimension nudges is keyed by registered option keys like `fix-now`, so 0 of 1,286 answers matched. Now five hook-captured label answers to the registered architecture-finding question move `architecture_care` from 0.5 to 0.632, and `--trace` lists all five.
+
+### Fixed
+- **`bin/gstack-question-log`**: on a hook-captured event (`source: hook`) for a registered `question_id`, the clicked label is mapped to that question's registered option key before it is written, and the label is kept in a new `choice_label` field. A label in `recommended` is mapped the same way, so `followed_recommendation` compares keys. Ad-hoc and `hook-<hash>` ids and unmatched or ambiguous labels are written as given. Rows from other writers (the agent's own log call, auto-decide, Codex import, typed free text) are unchanged, because readers such as the plan-ceo-review auto-decide check match the full mode names those rows carry. Existing logs are not rewritten.
+- **`scripts/psychographic-signals.ts`**: new `resolveOptionKey(question_id, label)`. It strips the `A)` selector and `(recommended)` suffix, then matches the label's leading words against the question's own options only: each key's own words (`fix-now` → "fix now") plus a short cue list ("Add to TODOs" → `defer`, "Ship as-is" → `skip`, "HOLD SCOPE" → `hold`). The longest cue wins and a tie stays unmapped.
+
+### Tests
+- `test/plan-tune.test.ts`: every registered option resolves from its own words, bare and as an `A) … (recommended)` label; 20 common phrasings, including the four plan-ceo-review mode names, map to their keys; negative controls (unregistered ids, a question with no options, a label naming no option, a mid-sentence cue, another question's key, a number that looks like a selector) stay unmapped. Two capture tests run the real bin: hook labels become keys with `choice_label` kept, while agent rows (including a `HOLD SCOPE` auto-decide row), ad-hoc ids, unmatched labels and free text are untouched, and a picked label follows a key recommendation. Both capture tests fail against the old bin.
+
+### Gate integrity
+- No test expectation, threshold, budget, timeout, trial count or case kind changed. Existing tests that pin label rows from the agent, auto-decide and Codex import paths pass unchanged.
+
 ## [1.91.222.0] - 2026-10-25
 
 **Five flaky tests now fail only for real defects, and a judge reply that isn't valid JSON says so in its failure line (daily flaky-fix pass, GSTA-91).** `context-restore-fragment-match` failed 0/3 on main cb99df4, although every run restored the right checkpoint: to find the checkpoint whose title matches, the agent often prints all three, and the case failed whenever another checkpoint's body appeared anywhere in the transcript. `test/gstack-ff-update.test.ts` counted every `sleep 300` on the machine, so heartbeats from parallel test files changed its count (remote pregate at #3412's head: expected "1", received "3"). The `GSTACK_CAPY=1` install-admission case in `test/gstack-skill-start.test.ts` makes ten `gstack-skill-start` spawns under bun's 5 s default timeout and took 3.6–5.7 s on a 4-core machine. The QA-only fixture's late scenarios mocked a 1000 ms capture budget but ran each attempt's real setup inside it. The setup-browser-cookies workflow judge in train-4 gate 40173759210 returned JSON with an unescaped quote in its reasoning, and the flake record read only "JSON Parse error: Expected '}'".

VERSION

@@ -1 +1 @@
-1.91.222.0
+1.91.223.0

agents-digest/gstack-AGENTS.md

@@ -1,4 +1,4 @@
-# gstack digest v1.91.222.0 — regenerate/re-copy after upgrading gstack
+# gstack digest v1.91.223.0 — regenerate/re-copy after upgrading gstack
 
 Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed
 for agent hosts without a full skill install. The full skills add workflows,

bin/gstack-question-log

@@ -18,8 +18,13 @@
 #   door_type          — one-way | two-way
 #                        (optional — looked up from registry if omitted)
 #   options_count      — number of options presented (positive integer)
-#   user_choice        — key user selected (free string; registry-options preferred)
-#   recommended        — option key the agent recommended (optional)
+#   user_choice        — key user selected (free string; registry-options preferred).
+#                        On hook-captured events (source=hook) for a registered
+#                        question_id, the clicked UI label is mapped to its
+#                        registered option key (#2623) and kept in choice_label.
+#                        Unmatched labels and other sources stay as given.
+#   recommended        — option key the agent recommended (optional; mapped
+#                        the same way on hook-captured events)
 #   followed_recommendation — bool (optional — computed if both present)
 #   session_id         — stable session identifier
 #   ts                 — ISO 8601 timestamp (auto-injected if missing)
@@ -46,8 +51,9 @@ INPUT="$1"
 TMPERR=$(mktemp "${TMPDIR:-/tmp}/gstack-question-log.XXXXXX") || { echo "gstack-question-log: could not create a temp file in ${TMPDIR:-/tmp}, so the question was not logged. Fix: point TMPDIR at a writable directory." >&2; exit 1; }
 trap 'rm -f "$TMPERR"' EXIT
 set +e
-VALIDATED=$(printf '%s' "$INPUT" | GSTACK_LIB_DIR="$SCRIPT_DIR/../lib" bun --no-env-file --config=/dev/null -e "
+VALIDATED=$(printf '%s' "$INPUT" | GSTACK_LIB_DIR="$SCRIPT_DIR/../lib" GSTACK_SCRIPTS_DIR="$SCRIPT_DIR/../scripts" bun --no-env-file --config=/dev/null -e "
 const { hasInjection } = await import(process.env.GSTACK_LIB_DIR + '/jsonl-store.ts');
+const { resolveOptionKey } = await import(process.env.GSTACK_SCRIPTS_DIR + '/psychographic-signals.ts');
 const path = require('path');
 const raw = await Bun.stdin.text();
 let j;
@@ -173,6 +179,23 @@ if (j.recommended !== undefined) {
   if (j.recommended.length > 64) j.recommended = j.recommended.slice(0, 64);
 }
 
+// Map the clicked UI label to its registered option key (#2623). SIGNAL_MAP
+// and question preferences are keyed by the registry's option keys, but the
+// PostToolUse hook only sees the label. Hook events and registered ids only;
+// an unmatched or ambiguous label stays as given. Agent-written rows keep
+// what the agent logged: skills such as plan-ceo-review log full mode names
+// that their own readers match on.
+if (j.source === 'hook') {
+  const key = resolveOptionKey(j.question_id, j.user_choice);
+  if (key !== undefined && key !== j.user_choice) {
+    j.choice_label = j.user_choice;
+    j.user_choice = key;
+  }
+  if (j.recommended !== undefined) {
+    j.recommended = resolveOptionKey(j.question_id, j.recommended) ?? j.recommended;
+  }
+}
+
 // followed_recommendation — compute if both sides present. An __unknown__
 // choice means extraction failed, not that the user rejected the
 // recommendation — leave the field absent so metrics can't be poisoned.

package.json

@@ -1,6 +1,6 @@
 {
   "name": "gstack",
-  "version": "1.91.222",
+  "version": "1.91.223",
   "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
   "license": "MIT",
   "type": "module",

scripts/psychographic-signals.ts

@@ -223,6 +223,89 @@ export const SIGNAL_MAP: Record<string, Record<string, DimensionDelta[]>> = {
   },
 };
 
+/**
+ * Label cues: option key → leading words of UI labels that mean that key.
+ *
+ * The AskUserQuestion hook only sees the label the user clicked ("A) Fix it
+ * now (recommended)"), never a stable key, while SIGNAL_MAP is keyed by the
+ * registry's option keys (#2623). resolveOptionKey() maps a label back to a
+ * key when bin/gstack-question-log records a hook event. Every key also
+ * matches its own words ("fix-now" → "fix now"), so only phrasings beyond
+ * that need an entry here. Cues are
+ * lowercase words separated by single spaces and match at the start of the
+ * normalized label; candidates are limited to the question's own registered
+ * options, and the longest cue wins.
+ */
+const OPTION_LABEL_CUES: Record<string, readonly string[]> = {
+  accept: ['yes', 'approve', 'agree', 'add', 'include', 'apply', 'adopt', 'enable', 'go ahead', 'do it', 'proceed', 'run', 'create', 'merge', 'roll back', 'migrate', 'open', 'promote', 'reorganize', 'polish'],
+  reject: ['no', 'disagree', 'decline', 'cancel', 'abort', 'dont', 'do not', 'keep current', 'keep as is', 'keep it as is'],
+  skip: ['no', 'no thanks', 'not now', 'dont', 'do not', 'pass', 'drop', 'ignore', 'leave it', 'ship as is', 'ship it as is'],
+  defer: ['later', 'fix later', 'add to todos', 'add to todo', 'add a todo', 'todo', 'postpone', 'backlog', 'park', 'track it'],
+  'fix-now': ['fix', 'fix it', 'fix them', 'fix before shipping'],
+  'fix-anyway': ['fix', 'fix it'],
+  investigate: ['dig in', 'look into', 'debug', 'diagnose', 'root cause'],
+  'ack-and-ship': ['ack', 'acknowledge', 'ship', 'ship anyway', 'ship it'],
+  'accept-risk': ['accept', 'accept the risk', 'live with it'],
+  'false-positive': ['not a bug', 'mark false positive', 'mark as false positive', 'mark it false positive'],
+  'add-test': ['add', 'add tests', 'add the test', 'write', 'write the test', 'test it'],
+  expand: ['scope expansion'],
+  selective: ['cherry pick', 'selective expansion'],
+  hold: ['keep', 'keep scope', 'hold scope'],
+  reduce: ['cut', 'trim', 'shrink', 'scope down', 'scope reduction'],
+  revise: ['edit', 'change', 'rework'],
+  restart: ['start over', 'redo'],
+  'build-now': ['build', 'build it now', 'include in this pr'],
+  'keep-local': ['keep'],
+  'keep-vendored': ['keep'],
+  'yes-upgrade': ['yes', 'upgrade'],
+  'always-auto': ['always'],
+  'not-now': ['later', 'snooze'],
+  'never-ask': ['never', 'disable'],
+};
+
+function normalizeOptionLabel(label: string): string {
+  return label
+    .toLowerCase()
+    .replace(/\s*\(recommended\)\s*$/, '')
+    .replace(/['\u2019]/g, '')
+    .replace(/^\s*(?:option\s+)?[a-z0-9]\s*(?:[).:]|-(?=\s))\s+/, '')
+    .replace(/[^a-z0-9]+/g, ' ')
+    .trim();
+}
+
+/**
+ * Map a chosen UI label to the registered option key of its question.
+ *
+ * Returns the key when exactly one of the question's registered options
+ * matches with the longest cue; undefined for unregistered ids, questions
+ * without options, unmatched labels and ties, so an unclear answer stays
+ * unmapped instead of nudging the wrong dimension.
+ */
+export function resolveOptionKey(question_id: string, choice: string): string | undefined {
+  const options = (QUESTIONS as Record<string, QuestionDef>)[question_id]?.options as string[] | undefined;
+  if (!options?.length) return undefined;
+  if (options.includes(choice)) return choice;
+  const label = normalizeOptionLabel(choice);
+  if (!label) return undefined;
+  let best: string | undefined;
+  let bestWords = 0;
+  let tie = false;
+  for (const key of options) {
+    for (const cue of [key.replace(/-/g, ' '), ...(OPTION_LABEL_CUES[key] ?? [])]) {
+      if (label !== cue && !label.startsWith(cue + ' ')) continue;
+      const words = cue.split(' ').length;
+      if (words > bestWords) {
+        best = key;
+        bestWords = words;
+        tie = false;
+      } else if (words === bestWords && best !== key) {
+        tie = true;
+      }
+    }
+  }
+  return tie ? undefined : best;
+}
+
 /**
  * Apply a user choice for a question to the running dimension totals.
  *

test/plan-tune.test.ts

@@ -33,6 +33,7 @@ import {
   validateRegistrySignalKeys,
   newDimensionTotals,
   normalizeToDimensionValue,
+  resolveOptionKey,
   ALL_DIMENSIONS,
 } from '../scripts/psychographic-signals';
 import {
@@ -290,6 +291,125 @@ describe('psychographic signal map', () => {
   });
 });
 
+// -----------------------------------------------------------------------
+// #2623 — chosen UI label → registered option key at capture time
+// -----------------------------------------------------------------------
+
+describe('option label → registered key (#2623)', () => {
+  const words = (key: string) => key.replace(/-/g, ' ');
+  const title = (s: string) => s.charAt(0).toUpperCase() + s.slice(1);
+
+  test('every registered option resolves from its own words, as a bare label and as an AUQ label', () => {
+    const misses: string[] = [];
+    for (const q of Object.values(QUESTIONS as Record<string, QuestionDef>)) {
+      for (const key of (q.options ?? []) as string[]) {
+        for (const label of [key, words(key), `A) ${title(words(key))} (recommended)`, `B. ${title(words(key))}`]) {
+          if (resolveOptionKey(q.id, label) !== key) misses.push(`${q.id}: ${JSON.stringify(label)} → ${resolveOptionKey(q.id, label)}`);
+        }
+      }
+    }
+    expect(misses).toEqual([]);
+  });
+
+  test('common phrasings map to the question\'s own key', () => {
+    const cases: Array<[string, string, string]> = [
+      ['plan-eng-review-arch-finding', 'A) Fix it now (recommended)', 'fix-now'],
+      ['plan-eng-review-arch-finding', 'Fix later', 'defer'],
+      ['plan-eng-review-arch-finding', 'B) Defer to TODOS.md', 'defer'],
+      ['plan-eng-review-arch-finding', 'C) Accept the risk and ship', 'accept-risk'],
+      ['plan-ceo-review-expansion-proposal', 'A) Add to plan (recommended)', 'accept'],
+      ['plan-ceo-review-expansion-proposal', 'B) Add to TODOs', 'defer'],
+      ['plan-ceo-review-mode', 'A) Expand scope (Recommended)', 'expand'],
+      ['plan-ceo-review-mode', 'A) SCOPE EXPANSION', 'expand'],
+      ['plan-ceo-review-mode', 'B) SELECTIVE EXPANSION', 'selective'],
+      ['plan-ceo-review-mode', 'C) HOLD SCOPE (recommended)', 'hold'],
+      ['plan-ceo-review-mode', 'D) SCOPE REDUCTION', 'reduce'],
+      ['plan-ceo-review-mode', 'Cherry-pick expansions', 'selective'],
+      ['plan-ceo-review-mode', 'Cut to the minimum', 'reduce'],
+      ['ship-pre-landing-review-fix', 'Ship as-is', 'skip'],
+      ['ship-test-failure-triage', 'Ship anyway, the failures are known', 'ack-and-ship'],
+      ['review-finding-fix', 'Not a bug, the input is trusted', 'false-positive'],
+      ['land-and-deploy-merge-confirm', 'Yes, merge it', 'accept'],
+      ['land-and-deploy-merge-confirm', 'Don’t merge yet', 'reject'],
+      ['plan-eng-review-test-gap', 'Add the test now', 'add-test'],
+      ['office-hours-mode-goal', 'Hackathon / demo', 'hackathon'],
+    ];
+    for (const [id, label, key] of cases) {
+      expect({ id, label, key: resolveOptionKey(id, label) }).toEqual({ id, label, key });
+    }
+  });
+
+  test('negative controls stay unmapped', () => {
+    // Unregistered (ad-hoc or hook-hash) ids: no registered options to map to.
+    expect(resolveOptionKey('plan-eng-review-ci-typecheck', 'Fix now')).toBeUndefined();
+    expect(resolveOptionKey('hook-0123456789', 'Accept')).toBeUndefined();
+    // Registered question with no options list.
+    expect(resolveOptionKey('plan-devex-review-persona', 'Accept')).toBeUndefined();
+    // A label that names none of the question's options.
+    expect(resolveOptionKey('plan-eng-review-arch-finding', 'Restore the parallel typecheck job')).toBeUndefined();
+    // Cues anchor at the start of the label, never mid-sentence.
+    expect(resolveOptionKey('review-sql-safety', 'Ship it, then fix')).toBeUndefined();
+    expect(resolveOptionKey('review-sql-safety', "Don't fix")).toBeUndefined();
+    // Another question's key is not a candidate.
+    expect(resolveOptionKey('qa-tier', 'Fix now')).toBeUndefined();
+    // A selector-looking number is not stripped into a match.
+    expect(resolveOptionKey('qa-tier', '1.5x deeper')).toBeUndefined();
+    expect(resolveOptionKey('plan-eng-review-arch-finding', '')).toBeUndefined();
+  });
+
+  function logAndRead(entries: Array<Record<string, unknown>>): Array<Record<string, unknown>> {
+    const tmpHome = fs.mkdtempSync(path.join(require('os').tmpdir(), 'gstack-2623-'));
+    try {
+      const { spawnSync } = require('child_process');
+      const env = { ...process.env, GSTACK_HOME: tmpHome, GSTACK_QUESTION_LOG_NO_DERIVE: '1' };
+      const logBin = path.join(ROOT, 'bin', 'gstack-question-log');
+      for (const e of entries) {
+        const r = spawnSync(logBin, [JSON.stringify({ skill: 'plan-eng-review', question_summary: 'q?', ...e })], { env, cwd: ROOT, encoding: 'utf-8', timeout: 30_000 });
+        expect(r.status).toBe(0);
+      }
+      const [slug] = fs.readdirSync(path.join(tmpHome, 'projects'));
+      return fs
+        .readFileSync(path.join(tmpHome, 'projects', slug, 'question-log.jsonl'), 'utf-8')
+        .trim()
+        .split('\n')
+        .map((l) => JSON.parse(l));
+    } finally {
+      fs.rmSync(tmpHome, { recursive: true, force: true });
+    }
+  }
+
+  test('gstack-question-log records the key and keeps the label for hook events on registered ids only', () => {
+    const [hook, agentKey, agentMode, adHoc, unmatched, freeText] = logAndRead([
+      { question_id: 'plan-eng-review-arch-finding', user_choice: 'A) Fix it now (recommended)', recommended: 'Fix it now', source: 'hook' },
+      { question_id: 'plan-eng-review-arch-finding', user_choice: 'defer', recommended: 'fix-now' },
+      { skill: 'plan-ceo-review', question_id: 'plan-ceo-review-mode', user_choice: 'HOLD SCOPE', recommended: 'HOLD SCOPE', auto_decided: true },
+      { question_id: 'plan-eng-review-ci-typecheck', user_choice: 'Fix now', source: 'hook' },
+      { question_id: 'plan-eng-review-arch-finding', user_choice: 'Restore the parallel typecheck job', source: 'hook' },
+      { question_id: 'plan-eng-review-arch-finding', user_choice: 'Fix it but rename the module first', free_text: 'Fix it but rename the module first', source: 'auq-other' },
+    ]);
+    expect(hook).toMatchObject({ user_choice: 'fix-now', choice_label: 'A) Fix it now (recommended)', recommended: 'fix-now', followed_recommendation: true });
+    expect(agentKey).toMatchObject({ user_choice: 'defer', recommended: 'fix-now', followed_recommendation: false });
+    expect(agentKey.choice_label).toBeUndefined();
+    // Agent-written rows keep what the agent logged; the auto-decide harness
+    // matches plan-ceo-review's full mode names.
+    expect(agentMode).toMatchObject({ source: 'agent', user_choice: 'HOLD SCOPE', recommended: 'HOLD SCOPE', followed_recommendation: true });
+    expect(agentMode.choice_label).toBeUndefined();
+    expect(adHoc).toMatchObject({ user_choice: 'Fix now' });
+    expect(adHoc.choice_label).toBeUndefined();
+    expect(unmatched).toMatchObject({ user_choice: 'Restore the parallel typecheck job' });
+    expect(unmatched.choice_label).toBeUndefined();
+    expect(freeText).toMatchObject({ user_choice: 'Fix it but rename the module first', source: 'auq-other' });
+    expect(freeText.choice_label).toBeUndefined();
+  });
+
+  test('a picked label scores as following a key recommendation', () => {
+    const [row] = logAndRead([
+      { question_id: 'plan-ceo-review-mode', user_choice: 'A) Expand scope (Recommended)', recommended: 'expand', source: 'hook' },
+    ]);
+    expect(row).toMatchObject({ user_choice: 'expand', followed_recommendation: true });
+  });
+});
+
 // -----------------------------------------------------------------------
 // Archetypes
 // -----------------------------------------------------------------------