diff --git a/docs/designs/PLAN_TUNING_V0.md b/docs/designs/PLAN_TUNING_V0.md index e9bf2b440..77bf95e8d 100644 --- a/docs/designs/PLAN_TUNING_V0.md +++ b/docs/designs/PLAN_TUNING_V0.md @@ -105,7 +105,7 @@ After weighing Codex's argument, we chose to roll back CEO EXPANSION and ship an } ``` -`signal_events` counts the logged answers that moved any dimension, and `contributing_events` counts, per dimension, the logged answers that moved it (an answer that writes two deltas to one dimension counts once), so a dimension with no evidence (0) is visible beside one whose evidence balances out; both print 0.5 (#2623). Each value is the sigmoid of its accumulated deltas, and each UTC day's sum per dimension (by the row's `ts`) is clamped to the largest single delta in the signal map (currently ±0.06), so one burst of answers can't pin a dimension; `--trace ` lists every contributing answer, each clamped day and the total. The bound keys on the day, not on `session_id`: the PostToolUse hook and the agent log different ids for the same review (the host's conversation id and the preamble's run id), and an agent can leave the `SESSION_ID` placeholder unsubstituted, so no logged id reliably marks one review. The cost is that several reviews on one day count as one. `contributing_events` still counts an answer logged by both writers twice. Only an ISO 8601 date or date-time `ts` counts as dated (a time without a zone is read as UTC), every other row shares one undated bucket, and `diversity.days_span` counts the same UTC days. Nothing displays `contributing_events` yet (#3436). +`signal_events` counts the logged answers that moved any dimension, and `contributing_events` counts, per dimension, the logged answers that moved it (an answer that writes two deltas to one dimension counts once), so a dimension with no evidence (0) is visible beside one whose evidence balances out; both print 0.5 (#2623). Each value is the sigmoid of its accumulated deltas, and each UTC day's sum per dimension (by the row's `ts`) is clamped to the largest single delta in the signal map (currently ±0.06), so one burst of answers can't pin a dimension; `--trace ` lists every contributing answer, each clamped day and the total. The bound keys on the day, not on `session_id`: the PostToolUse hook and the agent log different ids for the same review (the host's conversation id and the preamble's run id), and an agent can leave the `SESSION_ID` placeholder unsubstituted, so no logged id reliably marks one review. The cost is that several reviews on one day count as one. `contributing_events` still counts an answer logged by both writers twice. Only an ISO 8601 date or date-time `ts` counts as dated (a time without a zone is read as UTC), every other row shares one undated bucket, and `diversity.days_span` counts the same UTC days. `--check-mismatch`, `--gap`, the stored `gap` and /plan-tune's profile view compare a dimension only when its `contributing_events` is at least 3; `--gap` lists the others under `low_evidence` and the stored `gap` omits them (#3436). **Diversity check** (Codex #13): `inferred` is considered "enough data" only when `sample_size >= 20 AND skills_covered >= 3 AND question_ids_covered >= 8 AND days_span >= 7`. Below this, `/plan-tune profile` shows "not enough observed data yet" instead of a potentially-misleading inferred value. diff --git a/plan-tune/SKILL.md b/plan-tune/SKILL.md index eb13308fe..481d7764e 100644 --- a/plan-tune/SKILL.md +++ b/plan-tune/SKILL.md @@ -341,6 +341,8 @@ Parse the JSON. Present in **plain English**, not raw floats: the inferred column next to declared: "**scope_appetite:** declared 0.8 (boil the ocean) ↔ observed 0.72 (close)" Use words for the gap: 0.0-0.1 "close", 0.1-0.3 "drift", 0.3+ "mismatch". + When `inferred.contributing_events[dim]` is below 3, show "not enough + observed answers yet" for that dimension instead of an observed value or gap word. This display gate is intentionally lower than the E1 **promotion gate** (90+ days stable across 3+ skills, per `docs/designs/PLAN_TUNING_V0.md`). diff --git a/plan-tune/SKILL.md.tmpl b/plan-tune/SKILL.md.tmpl index 319ed6faa..88b7a8eb3 100644 --- a/plan-tune/SKILL.md.tmpl +++ b/plan-tune/SKILL.md.tmpl @@ -272,6 +272,8 @@ Parse the JSON. Present in **plain English**, not raw floats: the inferred column next to declared: "**scope_appetite:** declared 0.8 (boil the ocean) ↔ observed 0.72 (close)" Use words for the gap: 0.0-0.1 "close", 0.1-0.3 "drift", 0.3+ "mismatch". + When `inferred.contributing_events[dim]` is below 3, show "not enough + observed answers yet" for that dimension instead of an observed value or gap word. This display gate is intentionally lower than the E1 **promotion gate** (90+ days stable across 3+ skills, per `docs/designs/PLAN_TUNING_V0.md`).