mirror of
https://github.com/garrytan/gstack.git
synced 2026-05-02 03:35:09 +02:00
feat: opt-in usage telemetry + community intelligence platform (v0.8.6) (#210)
* feat: add gstack-telemetry-log and gstack-analytics scripts Local telemetry infrastructure for gstack usage tracking. gstack-telemetry-log appends JSONL events with skill name, duration, outcome, session ID, and platform info. Supports off/anonymous/community privacy tiers. gstack-analytics renders a personal usage dashboard from local data. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add telemetry preamble injection + opt-in prompt + epilogue Extends generatePreamble() with telemetry start block (config read, timer, session ID, .pending marker), opt-in prompt (gated by .telemetry-prompted), and epilogue instructions for Claude to log events after skill completion. Adds 5 telemetry tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate all SKILL.md files with telemetry blocks Automated regeneration from gen-skill-docs.ts changes. All skills now include telemetry start block, opt-in prompt, and epilogue. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add Supabase schema, edge functions, and SQL views Telemetry backend infrastructure: telemetry_events table with RLS (insert-only), installations table for retention tracking, update_checks for install pings. Edge functions for update-check (version + ping), telemetry-ingest (batch insert), and community-pulse (weekly active count). SQL views for crash clustering and skill co-occurrence sequences. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add telemetry-sync, community-dashboard, and integration tests gstack-telemetry-sync: fire-and-forget JSONL → Supabase sync with privacy tier field stripping, batch limits, and cursor tracking. gstack-community-dashboard: CLI tool querying Supabase for skill popularity, crash clusters, and version distribution. 19 integration tests covering all telemetry scripts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: session-specific .pending markers + crash_clusters view fix Addresses Codex review findings: - .pending race condition: use .pending-$SESSION_ID instead of shared .pending file to prevent concurrent session interference - crash_clusters view: add total_occurrences and anonymous_occurrences columns since anonymous tier has no installation_id - Added test: own session pending marker is not finalized Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: dual-attempt update check with Supabase install ping Fires a parallel background curl to Supabase during the slow-path version fetch. Logs upgrade_prompted event only on fresh fetches (not cached replays) to avoid overcounting. GitHub remains the primary version source — Supabase ping is fire-and-forget. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: integrate telemetry usage stats into /retro output Retro now reads ~/.gstack/analytics/skill-usage.jsonl and includes gstack usage metrics (skill run counts, top skills, success rate) in the weekly retrospective output. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: move 'Skill usage telemetry' to Completed in TODOS.md Implemented in this branch: local JSONL logging, opt-in prompt, privacy tiers, Supabase backend, community dashboard, /retro integration. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: wire Supabase credentials and expose tables via Data API Add supabase/config.sh with project URL and publishable key (safe to commit — RLS restricts to INSERT only). Update telemetry-sync, community-dashboard, and update-check to source the config and include proper auth headers for the Supabase REST API. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add SELECT RLS policies to migration for community dashboard reads All telemetry data is anonymous (no PII), so public reads via the publishable key are safe. Needed for the community dashboard to query skill popularity, crash clusters, and version distribution. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v0.8.6) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: analytics backward-compatible with old JSONL format Handle old-format events (no event_type field) alongside new format. Skip hook_fire events. Fix grep -c whitespace issues and unbound variable errors. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: map JSONL field names to Postgres columns in telemetry-sync Local JSONL uses short names (v, ts, sessions) but the Supabase table expects full names (schema_version, event_timestamp, concurrent_sessions). Add sed mapping during field stripping. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address Codex adversarial findings — cursor, opt-out, queries - Sync cursor now advances on HTTP 2xx (not grep for "inserted") - Update-check respects telemetry opt-out before pinging Supabase - Dashboard queries use correct view column names (total_occurrences) - Sync strips old-format "repo" field to prevent privacy leak Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add Privacy & Telemetry section to README Transparent disclosure of what telemetry collects, what it never sends, how to opt out, and a link to the schema so users can verify. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
Executable
+158
@@ -0,0 +1,158 @@
|
||||
#!/usr/bin/env bash
|
||||
# gstack-telemetry-log — append a telemetry event to local JSONL
|
||||
#
|
||||
# Data flow:
|
||||
# preamble (start) ──▶ .pending marker
|
||||
# preamble (epilogue) ──▶ gstack-telemetry-log ──▶ skill-usage.jsonl
|
||||
# └──▶ gstack-telemetry-sync (bg)
|
||||
#
|
||||
# Usage:
|
||||
# gstack-telemetry-log --skill qa --duration 142 --outcome success \
|
||||
# --used-browse true --session-id "12345-1710756600"
|
||||
#
|
||||
# Env overrides (for testing):
|
||||
# GSTACK_STATE_DIR — override ~/.gstack state directory
|
||||
# GSTACK_DIR — override auto-detected gstack root
|
||||
#
|
||||
# NOTE: Uses set -uo pipefail (no -e) — telemetry must never exit non-zero
|
||||
set -uo pipefail
|
||||
|
||||
GSTACK_DIR="${GSTACK_DIR:-$(cd "$(dirname "$0")/.." && pwd)}"
|
||||
STATE_DIR="${GSTACK_STATE_DIR:-$HOME/.gstack}"
|
||||
ANALYTICS_DIR="$STATE_DIR/analytics"
|
||||
JSONL_FILE="$ANALYTICS_DIR/skill-usage.jsonl"
|
||||
PENDING_DIR="$ANALYTICS_DIR" # .pending-* files live here
|
||||
CONFIG_CMD="$GSTACK_DIR/bin/gstack-config"
|
||||
VERSION_FILE="$GSTACK_DIR/VERSION"
|
||||
|
||||
# ─── Parse flags ─────────────────────────────────────────────
|
||||
SKILL=""
|
||||
DURATION=""
|
||||
OUTCOME="unknown"
|
||||
USED_BROWSE="false"
|
||||
SESSION_ID=""
|
||||
ERROR_CLASS=""
|
||||
EVENT_TYPE="skill_run"
|
||||
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--skill) SKILL="$2"; shift 2 ;;
|
||||
--duration) DURATION="$2"; shift 2 ;;
|
||||
--outcome) OUTCOME="$2"; shift 2 ;;
|
||||
--used-browse) USED_BROWSE="$2"; shift 2 ;;
|
||||
--session-id) SESSION_ID="$2"; shift 2 ;;
|
||||
--error-class) ERROR_CLASS="$2"; shift 2 ;;
|
||||
--event-type) EVENT_TYPE="$2"; shift 2 ;;
|
||||
*) shift ;;
|
||||
esac
|
||||
done
|
||||
|
||||
# ─── Read telemetry tier ─────────────────────────────────────
|
||||
TIER="$("$CONFIG_CMD" get telemetry 2>/dev/null || true)"
|
||||
TIER="${TIER:-off}"
|
||||
|
||||
# Validate tier
|
||||
case "$TIER" in
|
||||
off|anonymous|community) ;;
|
||||
*) TIER="off" ;; # invalid value → default to off
|
||||
esac
|
||||
|
||||
if [ "$TIER" = "off" ]; then
|
||||
# Still clear pending markers for this session even if telemetry is off
|
||||
[ -n "$SESSION_ID" ] && rm -f "$PENDING_DIR/.pending-$SESSION_ID" 2>/dev/null || true
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# ─── Finalize stale .pending markers ────────────────────────
|
||||
# Each session gets its own .pending-$SESSION_ID file to avoid races
|
||||
# between concurrent sessions. Finalize any that don't match our session.
|
||||
for PFILE in "$PENDING_DIR"/.pending-*; do
|
||||
[ -f "$PFILE" ] || continue
|
||||
# Skip our own session's marker (it's still in-flight)
|
||||
PFILE_BASE="$(basename "$PFILE")"
|
||||
PFILE_SID="${PFILE_BASE#.pending-}"
|
||||
[ "$PFILE_SID" = "$SESSION_ID" ] && continue
|
||||
|
||||
PENDING_DATA="$(cat "$PFILE" 2>/dev/null || true)"
|
||||
rm -f "$PFILE" 2>/dev/null || true
|
||||
if [ -n "$PENDING_DATA" ]; then
|
||||
# Extract fields from pending marker using grep -o + awk
|
||||
P_SKILL="$(echo "$PENDING_DATA" | grep -o '"skill":"[^"]*"' | head -1 | awk -F'"' '{print $4}')"
|
||||
P_TS="$(echo "$PENDING_DATA" | grep -o '"ts":"[^"]*"' | head -1 | awk -F'"' '{print $4}')"
|
||||
P_SID="$(echo "$PENDING_DATA" | grep -o '"session_id":"[^"]*"' | head -1 | awk -F'"' '{print $4}')"
|
||||
P_VER="$(echo "$PENDING_DATA" | grep -o '"gstack_version":"[^"]*"' | head -1 | awk -F'"' '{print $4}')"
|
||||
P_OS="$(uname -s | tr '[:upper:]' '[:lower:]')"
|
||||
P_ARCH="$(uname -m)"
|
||||
|
||||
# Write the stale event as outcome: unknown
|
||||
mkdir -p "$ANALYTICS_DIR"
|
||||
printf '{"v":1,"ts":"%s","event_type":"skill_run","skill":"%s","session_id":"%s","gstack_version":"%s","os":"%s","arch":"%s","duration_s":null,"outcome":"unknown","error_class":null,"used_browse":false,"sessions":1}\n' \
|
||||
"$P_TS" "$P_SKILL" "$P_SID" "$P_VER" "$P_OS" "$P_ARCH" >> "$JSONL_FILE" 2>/dev/null || true
|
||||
fi
|
||||
done
|
||||
|
||||
# Clear our own session's pending marker (we're about to log the real event)
|
||||
[ -n "$SESSION_ID" ] && rm -f "$PENDING_DIR/.pending-$SESSION_ID" 2>/dev/null || true
|
||||
|
||||
# ─── Collect metadata ────────────────────────────────────────
|
||||
TS="$(date -u +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || date -u +%Y-%m-%dT%H:%M:%S 2>/dev/null || echo "")"
|
||||
GSTACK_VERSION="$(cat "$VERSION_FILE" 2>/dev/null | tr -d '[:space:]' || echo "unknown")"
|
||||
OS="$(uname -s | tr '[:upper:]' '[:lower:]')"
|
||||
ARCH="$(uname -m)"
|
||||
SESSIONS="1"
|
||||
if [ -d "$STATE_DIR/sessions" ]; then
|
||||
_SC="$(find "$STATE_DIR/sessions" -mmin -120 -type f 2>/dev/null | wc -l | tr -d ' \n\r\t')"
|
||||
[ -n "$_SC" ] && [ "$_SC" -gt 0 ] 2>/dev/null && SESSIONS="$_SC"
|
||||
fi
|
||||
|
||||
# Generate installation_id for community tier
|
||||
INSTALL_ID=""
|
||||
if [ "$TIER" = "community" ]; then
|
||||
HOST="$(hostname 2>/dev/null || echo "unknown")"
|
||||
USER="$(whoami 2>/dev/null || echo "unknown")"
|
||||
if command -v shasum >/dev/null 2>&1; then
|
||||
INSTALL_ID="$(printf '%s-%s' "$HOST" "$USER" | shasum -a 256 | awk '{print $1}')"
|
||||
elif command -v sha256sum >/dev/null 2>&1; then
|
||||
INSTALL_ID="$(printf '%s-%s' "$HOST" "$USER" | sha256sum | awk '{print $1}')"
|
||||
elif command -v openssl >/dev/null 2>&1; then
|
||||
INSTALL_ID="$(printf '%s-%s' "$HOST" "$USER" | openssl dgst -sha256 | awk '{print $NF}')"
|
||||
fi
|
||||
# If no SHA-256 command available, install_id stays empty
|
||||
fi
|
||||
|
||||
# Local-only fields (never sent remotely)
|
||||
REPO_SLUG=""
|
||||
BRANCH=""
|
||||
if command -v git >/dev/null 2>&1; then
|
||||
REPO_SLUG="$(git remote get-url origin 2>/dev/null | sed 's|.*[:/]\([^/]*/[^/]*\)\.git$|\1|;s|.*[:/]\([^/]*/[^/]*\)$|\1|' | tr '/' '-' 2>/dev/null || true)"
|
||||
BRANCH="$(git rev-parse --abbrev-ref HEAD 2>/dev/null || true)"
|
||||
fi
|
||||
|
||||
# ─── Construct and append JSON ───────────────────────────────
|
||||
mkdir -p "$ANALYTICS_DIR"
|
||||
|
||||
# Escape null fields
|
||||
ERR_FIELD="null"
|
||||
[ -n "$ERROR_CLASS" ] && ERR_FIELD="\"$ERROR_CLASS\""
|
||||
|
||||
DUR_FIELD="null"
|
||||
[ -n "$DURATION" ] && DUR_FIELD="$DURATION"
|
||||
|
||||
INSTALL_FIELD="null"
|
||||
[ -n "$INSTALL_ID" ] && INSTALL_FIELD="\"$INSTALL_ID\""
|
||||
|
||||
BROWSE_BOOL="false"
|
||||
[ "$USED_BROWSE" = "true" ] && BROWSE_BOOL="true"
|
||||
|
||||
printf '{"v":1,"ts":"%s","event_type":"%s","skill":"%s","session_id":"%s","gstack_version":"%s","os":"%s","arch":"%s","duration_s":%s,"outcome":"%s","error_class":%s,"used_browse":%s,"sessions":%s,"installation_id":%s,"_repo_slug":"%s","_branch":"%s"}\n' \
|
||||
"$TS" "$EVENT_TYPE" "$SKILL" "$SESSION_ID" "$GSTACK_VERSION" "$OS" "$ARCH" \
|
||||
"$DUR_FIELD" "$OUTCOME" "$ERR_FIELD" "$BROWSE_BOOL" "${SESSIONS:-1}" \
|
||||
"$INSTALL_FIELD" "$REPO_SLUG" "$BRANCH" >> "$JSONL_FILE" 2>/dev/null || true
|
||||
|
||||
# ─── Trigger sync if tier is not off ─────────────────────────
|
||||
SYNC_CMD="$GSTACK_DIR/bin/gstack-telemetry-sync"
|
||||
if [ -x "$SYNC_CMD" ]; then
|
||||
"$SYNC_CMD" 2>/dev/null &
|
||||
fi
|
||||
|
||||
exit 0
|
||||
Reference in New Issue
Block a user