diff --git a/CHANGELOG.md b/CHANGELOG.md index 9008155b4..6680d44df 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,27 @@ # Changelog +## [1.91.233.0] - 2026-10-26 + +**/plan-tune's inferred profile no longer pins at 1.0 after a few reviews, and it now records how many answers moved each dimension.** This closes the last two parts of #2623. One /plan-eng-review produces about five answers, and every answer added its full delta, so eight sessions of "fix now" took `architecture_care` to 0.998, as uninformative as the 0.5 it replaced. Now each UTC day moves a dimension at most as far as one strong answer, and eight days of the same answers read 0.808. The profile JSON (`--profile`) also records `contributing_events` per dimension, so a dimension with no evidence reads 0 there instead of a bare 0.5; /plan-tune's display and `--check-mismatch` don't read it yet (#3436). + +### Fixed +- **`bin/gstack-developer-profile`**: `--derive` sums each UTC day's deltas per dimension (by the row's `ts`) and clamps the sum to the largest single delta in the signal map (±0.06 today). Answers within a day net out before the clamp. Only an ISO 8601 date or date-time `ts` counts as dated, and a time without a zone is read as UTC; every other row shares one undated bucket, so a junk timestamp can't escape the bound. `diversity.days_span` counts the same UTC days. The bound ignores `session_id` on purpose: the PostToolUse hook and the agent log different ids for one review, and an unsubstituted `SESSION_ID` placeholder would merge every review into one bound or leave each answer unbounded. Several reviews on one day count as one. `--trace` uses the same accumulation, so it lists every contributing answer, each clamped day ("day 2026-04-01: +0.250 capped to +0.060") and a `TOTAL` line whose value matches `--derive`. +- **`bin/gstack-developer-profile`**: a logged choice named after an `Object.prototype` member, such as `constructor`, or a log line that parses to a non-object such as `null`, no longer makes every later `--derive` and `--trace` fail; signal lookups use own properties only. `--trace` replaces control characters in the logged fields it prints, so a logged `ts` can't forge a `TOTAL` line. + +### Added +- **`inferred.contributing_events`**: per dimension, the number of logged answers that moved it (an answer that writes two deltas to one dimension counts once). Ported from #2625 by @v639dragoon, with co-author credit. #2625's autonomy signal from `followed_recommendation` stays parked, as decided in its Oct 8 review. + +### Docs +- `docs/designs/PLAN_TUNING_V0.md` documents `signal_events`, `contributing_events` and the per-day bound in the profile schema, including why the bound ignores `session_id`. +- `TODOS.md`: the #2623 entry now covers only its remaining Gate 1 work, the `{skill}-{slug}` hinted fallback, tracked as #3442. + +### Tests +- `test/gstack-developer-profile.test.ts`: `contributing_events` counts unique rows when one answer writes two dimensions, and is all zero for an answer that matches no signal. The bound equals the signal map's largest delta. Five answers on one day move a dimension by exactly the bound in both directions, whether they carry one `session_id`, five different ones, the literal `SESSION_ID` placeholder, or one review logged by both the hook and the agent. The day is the UTC day of `ts` even when the machine's zone is west of UTC, and a `ts` without a zone, `T`- or space-separated, is read as UTC. `--trace` prints the clamped days and a total that matches `--derive`, and replaces control characters. A `constructor` choice, a `null` line and array-valued `question_id` or `user_choice` rows are ignored. Rows whose `ts` is not ISO 8601 (`TS`, `junk 1`, a number) share one undated bucket and add no days to `days_span`. Negative controls: the same answers on separate days, a second day after a capped first day, and an answer pair that nets out are not clamped. The original positive tests fail against the old bin; the day-bound, `constructor` and control-character tests fail against the per-session version; a one-sided clamp fails the bound tests, and local-day bucketing fails the UTC tests. +- The test helper `logQuestion` sets `GSTACK_QUESTION_LOG_NO_DERIVE=1`, so the background derive can't race a test's own `--derive` (from #2625). + +### Gate integrity +- No test expectation, threshold, budget, timeout, trial count or case kind changed. + ## [1.91.232.0] - 2026-10-26 **The setup-browser-cookies workflow judge now has a preregistered calibration corpus, and its structured-output calibration is recorded as held, so the judge stays on today's request (GSTA-91).** In train-4 gate 40173759210, the cookie judge (claude-sonnet-5-5, free-text transport) returned JSON with an unescaped quote. The repo's fix for that shape is the JSON-schema transport, and `test/fixtures/judge-calibration/README.md` lets a configuration switch only after a calibration lands. The cookie configuration had no corpus until now. Its comparison-1 run removed the parse errors (old 2 of 144 samples, new 0 of 72) and matched or improved every other measure. It is held on one landing condition: a new false pass on a07. a07 scored 4/4/4 on all three new samples and on all three `old-b` samples. `old-a`'s a07 panel is incomplete because one old sample returned invalid JSON (both valid `old-a` a07 samples also scored 4/4/4), and false passes are counted against `old-a` alone, so the result counts against the new request only. diff --git a/TODOS.md b/TODOS.md index ed7010554..b2057075e 100644 --- a/TODOS.md +++ b/TODOS.md @@ -52,7 +52,7 @@ The 158 older open issues were re-checked against main 5cb5e1c (v1.91.83.0). Of Each item was decided in the Wave 9 plan and parked with its reason. Credit the reporter or contributor when it is picked up. - **Fold `triggers:` into skill descriptions (#2118)** — `scripts/gen-skill-docs.ts` folds only `voice-triggers:`, and the trimmed description keeps one sentence, so natural-language triggers never reach the listing. Parked because it conflicts with the catalog-trim design and would regenerate every skill. Mechanism when picked up: a `when_to_use:` frontmatter key for the Claude host, built from `triggers:`, deduped, and kept inside the catalog byte budget (Claude Code's 1,536-character combined listing limit); hosts that allowlist name and description get a bounded "Use when…" sentence or nothing. **Effort:** M. **Priority:** P2. -- **Plan-tune signals stuck at 0.5 (#2623)** — `hosts/claude/hooks/question-log-hook.ts` logs the UI label, while `scripts/psychographic-signals.ts` looks up stable option keys, so no row ever matches. The two-line key mapping is capture plumbing only and does not fix saturation. Acceptance criteria: labels map to stable option keys at log time; `profile.inferred` records contributing events per dimension; per-event and per-session accumulation is bounded so a dimension cannot pin at 1.0; `{skill}-{slug}` ids get a hinted fallback. Needs a design call on the key source. **Effort:** M. **Priority:** P2. +- **Plan-tune `{skill}-{slug}` ids never reach a signal (#3442, split from #2623)** — the label-to-key capture (#3394), `contributing_events` and the per-day bound (#3397) closed the rest of #2623. Remaining: `scripts/resolvers/question-tuning.ts` lets skills use `{skill}-{slug}` ids, which have no registry entry and so no `signal_key`; those answers never move a dimension. Acceptance: `{skill}-{slug}` ids get a hinted fallback to a registered signal key. Needs a design call on the key source. Credit @v639dragoon. **Effort:** M. **Priority:** P2. - **Stateless, instruction-only host mode (#3061, @m-check1B)** — hosted multi-tenant runtimes have no install, state root or helper binaries, but skill bodies still read and write `$GSTACK_STATE_ROOT`, call `bin/gstack-*` and log telemetry. Deferred to the new-hosts theme below. Requirements kept on record: tenant-owned state (a `capabilities.state: 'none'` host renders host-provided hooks instead of shell steps), named artifact hooks (`ARTIFACT_SAVE`/`ARTIFACT_FIND` per kind, `WEB_RESEARCH`, `SECRET_SCAN`, `USER_PREF`) mapped the way the gbrain resolvers already are, a generator check that fails a stateless render containing `~/.gstack`, `$GSTACK_`, `bin/gstack-*` or `--event-type`, and the `claude` host's output byte-identical. The contributor offered an office-hours overlay plus the generator check as a first PR. **Effort:** L. **Priority:** P3. - **Legacy unprefixed Cursor skill folders (#3084)** — 1.31 Cursor installs created `~/.cursor/skills//SKILL.md` links; 1.91 links `gstack-*` folders and `_prune_stale_generated` in `setup` only looks at `gstack-*` names, so Cursor can load two skills with the same name. `setup` already removes old unprefixed Claude entries; Cursor needs the same owned-link cleanup (remove only links that point into a gstack checkout). Split from #3084's data-loss fix, which Wave 9 Lane A0 ships. **Effort:** S. **Priority:** P3. - **`/retro global` says what a "session" is (#1315, third problem)** — `bin/gstack-global-discover.ts` counts transcript files and rollouts, so cron- or agent-driven repos show hundreds of "sessions" with no interactive work. Annotate the retro output that sessions are tool invocations or files, not distinct human working sessions. Problems 1 and 2 are fixed (v1.91.66.0 and Wave 9 Lane F). **Effort:** S. **Priority:** P3. diff --git a/VERSION b/VERSION index ca6a0eb0c..26e431ac8 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.91.232.0 +1.91.233.0 diff --git a/agents-digest/gstack-AGENTS.md b/agents-digest/gstack-AGENTS.md index d582ddc09..3b7189a1f 100644 --- a/agents-digest/gstack-AGENTS.md +++ b/agents-digest/gstack-AGENTS.md @@ -1,4 +1,4 @@ -# gstack digest v1.91.232.0 — regenerate/re-copy after upgrading gstack +# gstack digest v1.91.233.0 — regenerate/re-copy after upgrading gstack Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed for agent hosts without a full skill install. The full skills add workflows, diff --git a/bin/gstack-developer-profile b/bin/gstack-developer-profile index 5e004d453..b5db05262 100755 --- a/bin/gstack-developer-profile +++ b/bin/gstack-developer-profile @@ -11,7 +11,8 @@ # write updated ~/.gstack/developer-profile.json. # --profile emit the full profile as JSON (all fields). # --gap emit declared-vs-inferred gap as JSON. -# --trace show events that contributed to a dimension. +# --trace show events that contributed to a dimension, each +# UTC day clamped by the per-day bound, and the total. # --narrative (v2 stub) output a coach bio paragraph. # --vibe (v2 stub) output the one-word archetype. # --check-mismatch detect meaningful gaps between declared and observed. @@ -416,6 +417,79 @@ do_gap() { " } +# ----------------------------------------------------------------------- +# Signal accumulation shared by --derive and --trace, so the trace always +# explains exactly the value derive wrote. Each row's SIGNAL_MAP deltas are +# summed per (UTC day, dimension) and each day's sum is clamped to ±cap, +# where cap is the largest single delta in SIGNAL_MAP: one day moves a +# dimension at most as far as one strong answer. Without the bound a single +# /plan-eng-review (about five answers) moved a dimension by 0.25 and about +# eight reviews pinned it at 1.0 (#2623). The bound keys on the row's ts, +# not its session_id: writers log different ids for one review (the +# PostToolUse hook logs the host conversation, the agent the preamble run) +# and an unsubstituted placeholder such as SESSION_ID would merge or split +# reviews. Only an ISO 8601 date or date-time counts as dated; a time +# without a zone is read as UTC. Every other row shares one undated bucket, +# so junk timestamps cannot escape the bound (Date alone parses strings +# like "junk 1" as real days). diversity.days_span counts the same days. +# The body is a single-quoted bash string: its JS must not contain a single +# quote (use double quotes or backticks), or the assignment ends early and +# every subcommand fails to parse. +# ----------------------------------------------------------------------- +ACCUMULATE_SIGNALS_JS=' +function accumulateSignals(lines, QUESTIONS, SIGNAL_MAP, dims) { + let cap = 0; + for (const byChoice of Object.values(SIGNAL_MAP)) { + for (const deltas of Object.values(byChoice)) { + for (const d of deltas) cap = Math.max(cap, Math.abs(d.delta)); + } + } + const contributing = Object.fromEntries(dims.map((d) => [d, 0])); + const buckets = new Map(); + const rows = []; + const skills = new Set(); + const qids = new Set(); + const days = new Set(); + let count = 0; + let signalEvents = 0; + for (const line of lines) { + let e; + try { e = JSON.parse(line); } catch { continue; } + if (!e || typeof e !== "object" || typeof e.question_id !== "string" || typeof e.user_choice !== "string" || !e.question_id || !e.user_choice) continue; + count++; + skills.add(e.skill); + qids.add(e.question_id); + const m = typeof e.ts === "string" ? /^(\d{4}-\d\d-\d\d)(?:[Tt ](\d\d:\d\d(?::\d\d(?:\.\d+)?)?)(Z|z|[+-]\d\d:\d\d)?)?$/.exec(e.ts) : null; + const when = m ? new Date(m[2] ? m[1] + "T" + m[2] + (m[3] || "Z").toUpperCase() : m[1]) : null; + const day = when && !Number.isNaN(when.getTime()) ? when.toISOString().slice(0, 10) : null; + if (day) days.add(day); + const def = Object.hasOwn(QUESTIONS, e.question_id) ? QUESTIONS[e.question_id] : undefined; + const byChoice = def && def.signal_key && Object.hasOwn(SIGNAL_MAP, def.signal_key) ? SIGNAL_MAP[def.signal_key] : {}; + const deltas = Object.hasOwn(byChoice, e.user_choice) ? byChoice[e.user_choice] : []; + if (!Array.isArray(deltas) || deltas.length === 0) continue; + signalEvents++; + const key = day || "undated"; + const sums = buckets.get(key) || {}; + buckets.set(key, sums); + for (const d of deltas) { + sums[d.dim] = (sums[d.dim] || 0) + d.delta; + rows.push({ ts: e.ts, question_id: e.question_id, choice: e.user_choice, dim: d.dim, delta: d.delta }); + } + for (const dim of new Set(deltas.map((d) => d.dim))) contributing[dim] = (contributing[dim] ?? 0) + 1; + } + const totals = Object.fromEntries(dims.map((d) => [d, 0])); + const capped = []; + for (const [day, sums] of buckets) { + for (const [dim, sum] of Object.entries(sums)) { + const bounded = Math.max(-cap, Math.min(cap, sum)); + if (Math.abs(sum - bounded) > 1e-9) capped.push({ day, dim, sum, bounded }); + totals[dim] = (totals[dim] ?? 0) + bounded; + } + } + return { cap, totals, contributing, rows, capped, count, signalEvents, skills, qids, days }; +} +' + # ----------------------------------------------------------------------- # Derive: recompute inferred dimensions from question-events.jsonl # ----------------------------------------------------------------------- @@ -435,7 +509,8 @@ do_derive() { const sigmod = await import('./scripts/psychographic-signals.ts'); const fs = require('fs'); const { QUESTIONS } = regmod; - const { SIGNAL_MAP, applySignal, newDimensionTotals, normalizeToDimensionValue } = sigmod; + const { SIGNAL_MAP, ALL_DIMENSIONS, normalizeToDimensionValue } = sigmod; + $ACCUMULATE_SIGNALS_JS const profilePath = process.env.PROFILE_FILE_PATH; const eventsPath = process.env.EVENTS_PATH; @@ -446,25 +521,8 @@ do_derive() { lines = fs.readFileSync(eventsPath, 'utf-8').trim().split('\n').filter(Boolean); } - const totals = newDimensionTotals(); - const skills = new Set(); - const qids = new Set(); - const days = new Set(); - let count = 0; - let signalEvents = 0; - for (const line of lines) { - let e; - try { e = JSON.parse(line); } catch { continue; } - if (!e.question_id || !e.user_choice) continue; - count++; - skills.add(e.skill); - qids.add(e.question_id); - if (e.ts) days.add(String(e.ts).slice(0,10)); - const def = QUESTIONS[e.question_id]; - if (def && def.signal_key && applySignal(totals, def.signal_key, e.user_choice).length > 0) { - signalEvents++; - } - } + const { totals, contributing, count, signalEvents, skills, qids, days } = + accumulateSignals(lines, QUESTIONS, SIGNAL_MAP, ALL_DIMENSIONS); const values = {}; for (const [dim, total] of Object.entries(totals)) { @@ -477,6 +535,10 @@ do_derive() { // Events that actually moved a dimension (B10): sample_size counts every // logged answer, most of which match no signal and leave the 0.5 seed. signal_events: signalEvents, + // Unique rows that moved each dimension (#2625). Distinguishes 'no + // evidence' from 'balanced evidence': both print 0.5, which is how + // five 0.5s sat beside sample_size 1286 (#2623). + contributing_events: contributing, diversity: { skills_covered: skills.size, question_ids_covered: qids.size, @@ -522,29 +584,26 @@ do_trace() { const sigmod = await import('./scripts/psychographic-signals.ts'); const fs = require('fs'); const { QUESTIONS } = regmod; - const { SIGNAL_MAP } = sigmod; + const { SIGNAL_MAP, ALL_DIMENSIONS, normalizeToDimensionValue } = sigmod; + $ACCUMULATE_SIGNALS_JS const target = process.env.TRACE_DIM; const lines = fs.readFileSync(process.env.EVENTS_PATH, 'utf-8').trim().split('\n').filter(Boolean); - const rows = []; - for (const line of lines) { - let e; - try { e = JSON.parse(line); } catch { continue; } - const def = QUESTIONS[e.question_id]; - if (!def || !def.signal_key) continue; - const deltas = SIGNAL_MAP[def.signal_key]?.[e.user_choice] || []; - for (const d of deltas) { - if (d.dim === target) { - rows.push({ ts: e.ts, question_id: e.question_id, choice: e.user_choice, delta: d.delta }); - } - } - } + const acc = accumulateSignals(lines, QUESTIONS, SIGNAL_MAP, ALL_DIMENSIONS); + const rows = acc.rows.filter((r) => r.dim === target); + const signed = (n, digits) => (n > 0 ? '+' : '') + (digits === undefined ? String(n) : n.toFixed(digits)); + const clean = (s) => String(s ?? '').replace(/[\x00-\x1f\x7f-\x9f]/g, '?'); if (rows.length === 0) { console.log('TRACE: no events contribute to ' + target); } else { console.log('TRACE: ' + rows.length + ' events for ' + target); for (const r of rows) { - console.log(' ' + (r.ts || '').slice(0,19) + ' ' + r.question_id + ' → ' + r.choice + ' (' + (r.delta > 0 ? '+' : '') + r.delta + ')'); + console.log(' ' + clean(r.ts).slice(0,19) + ' ' + clean(r.question_id) + ' → ' + clean(r.choice) + ' (' + signed(r.delta) + ')'); } + for (const c of acc.capped.filter((c) => c.dim === target)) { + console.log(' day ' + clean(c.day) + ': ' + signed(c.sum, 3) + ' capped to ' + signed(c.bounded, 3)); + } + const total = acc.totals[target] ?? 0; + console.log('TOTAL: ' + signed(total, 3) + ' (each UTC day capped at ±' + acc.cap + ') → ' + normalizeToDimensionValue(total).toFixed(3)); } }); " diff --git a/docs/designs/PLAN_TUNING_V0.md b/docs/designs/PLAN_TUNING_V0.md index b1a0e7853..e9bf2b440 100644 --- a/docs/designs/PLAN_TUNING_V0.md +++ b/docs/designs/PLAN_TUNING_V0.md @@ -87,6 +87,8 @@ After weighing Codex's argument, we chose to roll back CEO EXPANSION and ship an "inferred": { "values": {"scope_appetite": 0.72, "risk_tolerance": 0.58, "...": "..."}, "sample_size": 47, + "signal_events": 19, + "contributing_events": {"scope_appetite": 9, "risk_tolerance": 6, "...": "..."}, "diversity": { "skills_covered": 5, "question_ids_covered": 14, @@ -103,6 +105,8 @@ After weighing Codex's argument, we chose to roll back CEO EXPANSION and ship an } ``` +`signal_events` counts the logged answers that moved any dimension, and `contributing_events` counts, per dimension, the logged answers that moved it (an answer that writes two deltas to one dimension counts once), so a dimension with no evidence (0) is visible beside one whose evidence balances out; both print 0.5 (#2623). Each value is the sigmoid of its accumulated deltas, and each UTC day's sum per dimension (by the row's `ts`) is clamped to the largest single delta in the signal map (currently ±0.06), so one burst of answers can't pin a dimension; `--trace ` lists every contributing answer, each clamped day and the total. The bound keys on the day, not on `session_id`: the PostToolUse hook and the agent log different ids for the same review (the host's conversation id and the preamble's run id), and an agent can leave the `SESSION_ID` placeholder unsubstituted, so no logged id reliably marks one review. The cost is that several reviews on one day count as one. `contributing_events` still counts an answer logged by both writers twice. Only an ISO 8601 date or date-time `ts` counts as dated (a time without a zone is read as UTC), every other row shares one undated bucket, and `diversity.days_span` counts the same UTC days. Nothing displays `contributing_events` yet (#3436). + **Diversity check** (Codex #13): `inferred` is considered "enough data" only when `sample_size >= 20 AND skills_covered >= 3 AND question_ids_covered >= 8 AND days_span >= 7`. Below this, `/plan-tune profile` shows "not enough observed data yet" instead of a potentially-misleading inferred value. ## Data flow (v1) diff --git a/package.json b/package.json index 5302ec594..b4a7ae0e4 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gstack", - "version": "1.91.232", + "version": "1.91.233", "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.", "license": "MIT", "type": "module", diff --git a/test/gstack-developer-profile.test.ts b/test/gstack-developer-profile.test.ts index 0550dfc31..11685bb64 100644 --- a/test/gstack-developer-profile.test.ts +++ b/test/gstack-developer-profile.test.ts @@ -16,6 +16,7 @@ import * as fs from 'fs'; import * as path from 'path'; import * as os from 'os'; import { spawnSync } from 'child_process'; +import { SIGNAL_MAP, normalizeToDimensionValue } from '../scripts/psychographic-signals'; const ROOT = path.resolve(import.meta.dir, '..'); const BIN_DEV = path.join(ROOT, 'bin', 'gstack-developer-profile'); @@ -47,7 +48,9 @@ function runDev(...args: string[]): { stdout: string; stderr: string; status: nu function logQuestion(payload: Record): number { const res = spawnSync(BIN_LOG, [JSON.stringify(payload)], { - env: { ...process.env, GSTACK_HOME: tmpHome }, + // gstack-question-log fires `--derive` in the background after every + // append, which races the explicit runDev('--derive') a test then makes. + env: { ...process.env, GSTACK_HOME: tmpHome, GSTACK_QUESTION_LOG_NO_DERIVE: '1' }, encoding: 'utf-8', cwd: ROOT, timeout: 30_000, @@ -308,6 +311,197 @@ describe('gstack-developer-profile --derive', () => { expect(p.inferred.sample_size).toBe(1); expect(p.inferred.values.scope_appetite).toBeCloseTo(0.5, 2); }); + + test('contributing_events counts unique rows per dimension (#2625)', () => { + // plan-eng-review-arch-finding → fix-now writes architecture_care AND + // risk_tolerance from one row; an ad-hoc row writes nothing. + for (let i = 0; i < 2; i++) { + logQuestion({ skill: 'plan-eng-review', question_id: 'plan-eng-review-arch-finding', question_summary: 'arch?', user_choice: 'fix-now', session_id: `s${i}` }); + } + logQuestion({ skill: 'plan-eng-review', question_id: 'adhoc-unregistered-question', question_summary: 'x', user_choice: 'fix-now', session_id: 's9' }); + runDev('--derive'); + const p = readProfile() as { inferred: { contributing_events: Record; signal_events: number } }; + expect(p.inferred.contributing_events).toEqual({ + scope_appetite: 0, + risk_tolerance: 2, + detail_preference: 0, + autonomy: 0, + architecture_care: 2, + }); + expect(p.inferred.signal_events).toBe(2); + }); + + test('contributing_events is all zero when no answer matches a signal', () => { + logQuestion({ skill: 'plan-eng-review', question_id: 'plan-eng-review-arch-finding', question_summary: 'arch?', user_choice: 'not-a-mapped-option', session_id: 's1' }); + runDev('--derive'); + const p = readProfile() as { inferred: { contributing_events: Record; sample_size: number } }; + expect(p.inferred.sample_size).toBe(1); + expect(p.inferred.contributing_events).toEqual({ + scope_appetite: 0, + risk_tolerance: 0, + detail_preference: 0, + autonomy: 0, + architecture_care: 0, + }); + }); + + test('a choice named after an Object.prototype member or a non-object line is ignored, not fatal', () => { + logQuestion({ skill: 'plan-eng-review', question_id: 'plan-eng-review-arch-finding', question_summary: 'arch?', user_choice: 'constructor', session_id: 's1' }); + logQuestion({ skill: 'plan-eng-review', question_id: 'plan-eng-review-arch-finding', question_summary: 'arch?', user_choice: 'fix-now', session_id: 's2' }); + const projects = path.join(tmpHome, 'projects'); + for (const slug of fs.readdirSync(projects)) { + const log = path.join(projects, slug, 'question-log.jsonl'); + if (!fs.existsSync(log)) continue; + fs.appendFileSync(log, 'null\n'); + fs.appendFileSync(log, JSON.stringify({ skill: 'plan-eng-review', question_id: ['plan-eng-review-arch-finding'], user_choice: 'fix-now' }) + '\n'); + fs.appendFileSync(log, JSON.stringify({ skill: 'plan-eng-review', question_id: 'plan-eng-review-arch-finding', user_choice: ['fix-now'] }) + '\n'); + } + const r = runDev('--derive'); + expect(r.status).toBe(0); + const p = readProfile() as { inferred: { contributing_events: Record; signal_events: number; sample_size: number } }; + expect(p.inferred.sample_size).toBe(2); + expect(p.inferred.signal_events).toBe(1); + expect(p.inferred.contributing_events.architecture_care).toBe(1); + expect(runDev('--trace', 'architecture_care').stdout).toContain('1 events for architecture_care'); + }); +}); + +// ----------------------------------------------------------------------- +// Per-day bound (#2623): one UTC day moves a dimension at most as far as the +// largest single delta in SIGNAL_MAP, whatever session_id the writers log. +// ----------------------------------------------------------------------- + +describe('gstack-developer-profile per-day bound', () => { + const CAP = Math.max(...Object.values(SIGNAL_MAP).flatMap((byChoice) => Object.values(byChoice).flat().map((d) => Math.abs(d.delta)))); + const arch = (day: number, user_choice = 'fix-now', extra: Record = {}) => + logQuestion({ skill: 'plan-eng-review', question_id: 'plan-eng-review-arch-finding', question_summary: 'arch?', user_choice, ts: `2026-04-${String(day).padStart(2, '0')}T10:00:00Z`, ...extra }); + const inferred = () => (readProfile() as { inferred: { values: Record; contributing_events: Record } }).inferred; + const value = (total: number) => +normalizeToDimensionValue(total).toFixed(3); + + test('the bound is the largest single delta in the signal map', () => { + expect(CAP).toBe(0.06); + }); + + test('five answers on one day move a dimension by the bound, not by their sum', () => { + for (let i = 0; i < 5; i++) arch(1, 'fix-now', { session_id: 'one-review' }); + runDev('--derive'); + expect(inferred().values.architecture_care).toBe(value(CAP)); + expect(inferred().values.risk_tolerance).toBe(value(-CAP)); + expect(inferred().contributing_events.architecture_care).toBe(5); + }); + + test('the same five answers on five days are not bounded (negative control)', () => { + for (let i = 0; i < 5; i++) arch(i + 1, 'fix-now', { session_id: 'one-review' }); + runDev('--derive'); + expect(inferred().values.architecture_care).toBe(value(5 * 0.05)); + }); + + test('a second day still moves the dimension after the first day is capped (negative control)', () => { + for (let i = 0; i < 5; i++) arch(1); + for (let i = 0; i < 5; i++) arch(2); + runDev('--derive'); + expect(inferred().values.architecture_care).toBe(value(2 * CAP)); + }); + + test('distinct session_ids on one day share the day bound', () => { + for (let i = 0; i < 5; i++) arch(1, 'fix-now', { session_id: `s${i}` }); + runDev('--derive'); + expect(inferred().values.architecture_care).toBe(value(CAP)); + }); + + test('an unsubstituted SESSION_ID placeholder is bounded per day like any other id', () => { + for (let i = 0; i < 5; i++) arch(1, 'fix-now', { session_id: 'SESSION_ID' }); + for (let i = 0; i < 3; i++) arch(i + 2, 'fix-now', { session_id: 'SESSION_ID' }); + runDev('--derive'); + expect(inferred().values.architecture_care).toBe(value(CAP + 3 * 0.05)); + }); + + test('one review logged by both the hook and the agent is bounded once per day', () => { + for (let i = 0; i < 5; i++) { + arch(1, 'fix-now', { source: 'hook', tool_use_id: `t${i}`, session_id: 'claude-conversation-uuid' }); + arch(1, 'fix-now', { session_id: '1234-1700000000-ab' }); + } + runDev('--derive'); + expect(inferred().values.architecture_care).toBe(value(CAP)); + expect(inferred().contributing_events.architecture_care).toBe(10); + }); + + const deriveWestOfUtc = () => { + const res = spawnSync(BIN_DEV, ['--derive'], { + env: { ...process.env, GSTACK_HOME: tmpHome, TZ: 'America/Los_Angeles' }, + encoding: 'utf-8', + cwd: ROOT, + timeout: 30_000, + }); + expect(res.status).toBe(0); + }; + const atTs = (ts: unknown) => + logQuestion({ skill: 'plan-eng-review', question_id: 'plan-eng-review-arch-finding', question_summary: 'arch?', user_choice: 'fix-now', ts }); + + test('the day is the UTC day of ts, not the local day', () => { + arch(1, 'fix-now'); + atTs('2026-04-01T20:00:00-08:00'); + deriveWestOfUtc(); + expect(inferred().values.architecture_care).toBe(value(2 * 0.05)); + expect((readProfile() as { inferred: { diversity: { days_span: number } } }).inferred.diversity.days_span).toBe(2); + }); + + test('a ts without a zone is read as UTC', () => { + atTs('2026-04-01T23:30:00'); + atTs('2026-04-02T00:30:00Z'); + deriveWestOfUtc(); + expect(inferred().values.architecture_care).toBe(value(2 * 0.05)); + expect((readProfile() as { inferred: { diversity: { days_span: number } } }).inferred.diversity.days_span).toBe(2); + }); + + test('a ts with a space separator and no zone is read as UTC', () => { + atTs('2026-04-01 23:30:00'); + atTs('2026-04-02T00:30:00Z'); + deriveWestOfUtc(); + expect(inferred().values.architecture_care).toBe(value(2 * 0.05)); + expect((readProfile() as { inferred: { diversity: { days_span: number } } }).inferred.diversity.days_span).toBe(2); + }); + + test('rows without an ISO 8601 string ts share one undated bucket', () => { + const junk: unknown[] = ['not-a-date', 'TS', 'junk 1', 'foo 12', 1775000000]; + for (const ts of junk) expect(atTs(ts)).toBe(0); + runDev('--derive'); + const p = readProfile() as { inferred: { sample_size: number; contributing_events: Record; diversity: { days_span: number } } }; + expect(p.inferred.sample_size).toBe(junk.length); + expect(p.inferred.contributing_events.architecture_care).toBe(junk.length); + expect(inferred().values.architecture_care).toBe(value(CAP)); + expect(p.inferred.diversity.days_span).toBe(0); + }); + + test('answers on one day net out before the bound applies', () => { + arch(1, 'fix-now'); + arch(1, 'accept-risk'); + runDev('--derive'); + expect(inferred().values.architecture_care).toBe(value(0.05 - 0.04)); + }); + + test('--trace shows the capped day and a total that matches --derive', () => { + for (let i = 0; i < 5; i++) arch(1); + arch(7, 'defer'); + runDev('--derive'); + const r = runDev('--trace', 'architecture_care'); + expect(r.stdout).toContain('6 events for architecture_care'); + expect(r.stdout).toContain('day 2026-04-01: +0.250 capped to +0.060'); + expect(r.stdout).not.toContain('day 2026-04-07'); + expect(r.stdout).toContain(`TOTAL: +0.040 (each UTC day capped at ±${CAP}) → ${inferred().values.architecture_care.toFixed(3)}`); + const risk = runDev('--trace', 'risk_tolerance'); + expect(risk.stdout).toContain('day 2026-04-01: -0.100 capped to -0.060'); + }); + + test('--trace replaces control characters in logged fields', () => { + for (let i = 0; i < 2; i++) { + logQuestion({ skill: 'plan-eng-review', question_id: 'plan-eng-review-arch-finding', question_summary: 'arch?', user_choice: 'fix-now', ts: '\u001b\u007f\u009b\nTOTAL: +9.999' }); + } + const r = runDev('--trace', 'architecture_care'); + expect(r.stdout.split('\n').filter((l) => l.startsWith('TOTAL:'))).toHaveLength(1); + expect(r.stdout).toContain(' ????TOTAL: +9.999 plan-eng-review-arch-finding'); + expect(/[\x00-\x08\x0b-\x1f\x7f-\x9f]/.test(r.stdout)).toBe(false); + }); }); // -----------------------------------------------------------------------