From fb950ad7b7d1a763b4725460bab5874823442c4d Mon Sep 17 00:00:00 2001 From: garrytan Date: Mon, 2 Nov 2026 13:36:42 +0000 Subject: [PATCH] docs(plan-tune): document signal hints and hinted_events in the design reference (#3442) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --- docs/designs/PLAN_TUNING_V0.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/designs/PLAN_TUNING_V0.md b/docs/designs/PLAN_TUNING_V0.md index 77bf95e8d..523c31044 100644 --- a/docs/designs/PLAN_TUNING_V0.md +++ b/docs/designs/PLAN_TUNING_V0.md @@ -89,6 +89,7 @@ After weighing Codex's argument, we chose to roll back CEO EXPANSION and ship an "sample_size": 47, "signal_events": 19, "contributing_events": {"scope_appetite": 9, "risk_tolerance": 6, "...": "..."}, + "hinted_events": 3, "diversity": { "skills_covered": 5, "question_ids_covered": 14, @@ -105,7 +106,7 @@ After weighing Codex's argument, we chose to roll back CEO EXPANSION and ship an } ``` -`signal_events` counts the logged answers that moved any dimension, and `contributing_events` counts, per dimension, the logged answers that moved it (an answer that writes two deltas to one dimension counts once), so a dimension with no evidence (0) is visible beside one whose evidence balances out; both print 0.5 (#2623). Each value is the sigmoid of its accumulated deltas, and each UTC day's sum per dimension (by the row's `ts`) is clamped to the largest single delta in the signal map (currently ±0.06), so one burst of answers can't pin a dimension; `--trace ` lists every contributing answer, each clamped day and the total. The bound keys on the day, not on `session_id`: the PostToolUse hook and the agent log different ids for the same review (the host's conversation id and the preamble's run id), and an agent can leave the `SESSION_ID` placeholder unsubstituted, so no logged id reliably marks one review. The cost is that several reviews on one day count as one. `contributing_events` still counts an answer logged by both writers twice. Only an ISO 8601 date or date-time `ts` counts as dated (a time without a zone is read as UTC), every other row shares one undated bucket, and `diversity.days_span` counts the same UTC days. `--check-mismatch`, `--gap`, the stored `gap` and /plan-tune's profile view compare a dimension only when its `contributing_events` is at least 3; `--gap` lists the others under `low_evidence` and the stored `gap` omits them (#3436). +`signal_events` counts the logged answers that moved any dimension, and `contributing_events` counts, per dimension, the logged answers that moved it (an answer that writes two deltas to one dimension counts once), so a dimension with no evidence (0) is visible beside one whose evidence balances out; both print 0.5 (#2623). Each value is the sigmoid of its accumulated deltas, and each UTC day's sum per dimension (by the row's `ts`) is clamped to the largest single delta in the signal map (currently ±0.06), so one burst of answers can't pin a dimension; `--trace ` lists every contributing answer, each clamped day and the total. The bound keys on the day, not on `session_id`: the PostToolUse hook and the agent log different ids for the same review (the host's conversation id and the preamble's run id), and an agent can leave the `SESSION_ID` placeholder unsubstituted, so no logged id reliably marks one review. The cost is that several reviews on one day count as one. `contributing_events` still counts an answer logged by both writers twice. Only an ISO 8601 date or date-time `ts` counts as dated (a time without a zone is read as UTC), every other row shares one undated bucket, and `diversity.days_span` counts the same UTC days. `--check-mismatch`, `--gap`, the stored `gap` and /plan-tune's profile view compare a dimension only when its `contributing_events` is at least 3; `--gap` lists the others under `low_evidence` and the stored `gap` omits them (#3436). An unregistered `{skill}-{slug}` id moves a dimension only through a signal hint (#3442): the asked question carries `` and the log row a matching `signal_key`, chosen from `gstack-question-preference --signal-keys`. Derive uses the hint only when the id has no registry entry, the key is in the signal map and the answer matches one of that signal's option keys; `hinted_events` counts those signal events, and `--trace` marks their rows `(hinted)`. **Diversity check** (Codex #13): `inferred` is considered "enough data" only when `sample_size >= 20 AND skills_covered >= 3 AND question_ids_covered >= 8 AND days_span >= 7`. Below this, `/plan-tune profile` shows "not enough observed data yet" instead of a potentially-misleading inferred value.