Telemetry records why skill runs end, which host and model ran them, and how upgrades went (GST-3234) (#4542)

gstack's anonymous usage telemetry now records nine more fields: the
host and model family, how a skill was started, why it ended, whether
checks passed, how skill routing is set up, a random per-session id, and
for upgrades the version left and any error.

This tells us where skills stop working and which setups break upgrades,
which the current outcome-only data can't show.

Deploy in order after merge: apply migration 009, deploy the
telemetry-ingest function, then release the client; apply migration 007
after that.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Review-Finding: p1/specialist-batched#2 reader=specialist-batched
Review-Finding: p1/specialist-batched#3 reader=specialist-batched
Review-Finding: p1/adversarial-native#1 reader=adversarial-native
Review-Finding: p1/outside-voice#2 reader=outside-voice
Review-Finding: v1/outside-voice#1 reader=outside-voice
Review-Finding: v1/outside-voice#2 reader=outside-voice
This commit is contained in:
Garry Tanandcapy-ai[bot] authored and GitHub committed 2026-11-19 23:10:19 +00:00
1 parent db2a56e679
commit 9ea26ecc42
74 files changed
+1180 -695

No files matched your search

@@ -69,24 +69,24 @@ Do not log obvious facts or one-time transient errors.
## Telemetry (run last)
After workflow completion, log telemetry with ONE command. OUTCOME is
success/error/abort/unknown; \`SESSION_ID\` and \`TEL_START\` are the values the
preamble's skill-start output echoed. It also drains the artifacts-sync queue
(the former skill-end sync step — do not run gstack-brain-sync separately).
When done, make ONE call — it also drains the artifacts-sync
queue (don't also run gstack-brain-sync).
**PLAN MODE EXCEPTION — ALWAYS RUN:** This writes telemetry to
\`$GSTACK_STATE_ROOT/analytics/\`, matching preamble analytics writes.
**PLAN MODE EXCEPTION — ALWAYS RUN:** writes telemetry to \`$GSTACK_STATE_ROOT/analytics/\`.
\`\`\`bash
${ctx.paths.binDir}/gstack-skill-end --skill "${ctx.skillName}" --outcome OUTCOME \\
--session-id "SESSION_ID" --tel-start "TEL_START" --used-browse USED_BROWSE \\
--error-message "ERROR_MESSAGE" --failed-step "FAILED_STEP" 2>/dev/null || true
--error-message "ERROR_MESSAGE" --failed-step "FAILED_STEP" \\
--exit-reason EXIT_REASON --invoked INVOKED --model-family MODEL_FAMILY --checks CHECKS 2>/dev/null || true
\`\`\`
Replace \`OUTCOME\` and \`USED_BROWSE\` (yes/no) before running; substitute
\`SESSION_ID\`/\`TEL_START\` from the skill-start echoes. \`ERROR_MESSAGE\`/\`FAILED_STEP\`
are "" unless outcome is error. If the command is missing (stale install), skip
telemetry — it never blocks the workflow.
OUTCOME: success/error/abort/unknown. USED_BROWSE: yes/no. SESSION_ID, TEL_START:
skill-start echoes. ERROR_MESSAGE, FAILED_STEP: "" unless error. EXIT_REASON:
completed/user_stopped/blocked/tool_error/test_failed/context_limit/gave_up.
INVOKED: user/model/chained. MODEL_FAMILY (yours): claude_opus/claude_sonnet/
claude_haiku/claude_other/gpt/gemini/other. CHECKS: passed/failed/not_run. No command
(stale install) — skip.
## Plan Status Footer