mirror of
https://github.com/garrytan/gstack.git
synced 2026-05-01 19:25:10 +02:00
fix: Supabase telemetry security lockdown (v0.11.16.0) (#460)
* fix: drop all anon RLS policies + revoke view access + add cache table Migration 002 locks down the Supabase telemetry backend: - Drops all SELECT, INSERT, UPDATE policies for the anon role - Explicitly revokes SELECT on crash_clusters and skill_sequences views - Drops stale error_message/failed_step columns (exist live but not in migration) - Creates community_pulse_cache table for server-side aggregation caching * feat: extend community-pulse with full dashboard data + server-side cache community-pulse now returns top skills, crash clusters, version distribution, and weekly active count in a single aggregated response. Results are cached in the community_pulse_cache table (1-hour TTL) to prevent DoS via repeated expensive queries. * fix: route all telemetry through edge functions, not PostgREST - gstack-telemetry-sync: POST to /functions/v1/telemetry-ingest instead of /rest/v1/telemetry_events. Removes sed field-renaming (edge function expects raw JSONL names). Parses inserted count — holds cursor if zero inserted. - gstack-update-check: POST to /functions/v1/update-check. - gstack-community-dashboard: calls community-pulse edge function instead of direct PostgREST queries. - config.sh: removes GSTACK_TELEMETRY_ENDPOINT, fixes misleading comment. * test: RLS smoke test + telemetry field name verification - verify-rls.sh: 9-check smoke test (5 reads + 3 inserts + 1 update) verifying anon key is fully locked out after migration. - telemetry.test.ts: verifies JSONL uses raw field names (v, ts, sessions) that the edge function expects, not Postgres column names. - README.md: fixes privacy claim to match actual RLS policy. * chore: bump version and changelog (v0.11.16.0) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: pre-landing review fixes — JSONB field order, version filter, RLS verification - Dashboard JSON parsing: use per-object grep instead of field-order-dependent regex (JSONB doesn't preserve key order) - Version distribution: filter to skill_run events only (was counting all types) - verify-rls.sh: only 401/403 count as PASS (not empty 200 or 5xx); add Authorization header to test as anon role properly - Remove dead empty loop in community-pulse * chore: untrack browse/dist binaries — 116MB of arm64-only Mach-O These compiled Bun binaries only work on arm64 macOS, and ./setup already rebuilds from source for every platform. They were tracked despite .gitignore due to being committed before the ignore rule. Untracking stops them from appearing as modified in every diff. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: tone down changelog — security hardening, not incident report * fix: keep INSERT policies for old client compat, preserve extra columns - Keep anon INSERT policies so pre-v0.11.16 clients can still sync telemetry via PostgREST while new clients use edge functions - Add error_message/failed_step columns to migration (reconcile repo with live schema) instead of dropping them - Security fix still lands: SELECT and UPDATE policies are dropped Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: sync package.json version with VERSION file (0.11.16.0) --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env bash
|
||||
# gstack-community-dashboard — community usage stats from Supabase
|
||||
#
|
||||
# Queries the Supabase REST API to show community-wide gstack usage:
|
||||
# Calls the community-pulse edge function for aggregated stats:
|
||||
# skill popularity, crash clusters, version distribution, retention.
|
||||
#
|
||||
# Env overrides (for testing):
|
||||
@@ -30,51 +30,40 @@ if [ -z "$SUPABASE_URL" ] || [ -z "$ANON_KEY" ]; then
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# ─── Helper: query Supabase REST API ─────────────────────────
|
||||
query() {
|
||||
local table="$1"
|
||||
local params="${2:-}"
|
||||
curl -sf --max-time 10 \
|
||||
"${SUPABASE_URL}/rest/v1/${table}?${params}" \
|
||||
-H "apikey: ${ANON_KEY}" \
|
||||
-H "Authorization: Bearer ${ANON_KEY}" \
|
||||
2>/dev/null || echo "[]"
|
||||
}
|
||||
# ─── Fetch aggregated stats from edge function ────────────────
|
||||
DATA="$(curl -sf --max-time 15 \
|
||||
"${SUPABASE_URL}/functions/v1/community-pulse" \
|
||||
-H "apikey: ${ANON_KEY}" \
|
||||
2>/dev/null || echo "{}")"
|
||||
|
||||
echo "gstack community dashboard"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
echo ""
|
||||
|
||||
# ─── Weekly active installs ──────────────────────────────────
|
||||
WEEK_AGO="$(date -u -v-7d +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || date -u -d '7 days ago' +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || echo "")"
|
||||
if [ -n "$WEEK_AGO" ]; then
|
||||
PULSE="$(curl -sf --max-time 10 \
|
||||
"${SUPABASE_URL}/functions/v1/community-pulse" \
|
||||
-H "Authorization: Bearer ${ANON_KEY}" \
|
||||
2>/dev/null || echo '{"weekly_active":0}')"
|
||||
WEEKLY="$(echo "$DATA" | grep -o '"weekly_active":[0-9]*' | grep -o '[0-9]*' || echo "0")"
|
||||
CHANGE="$(echo "$DATA" | grep -o '"change_pct":[0-9-]*' | grep -o '[0-9-]*' || echo "0")"
|
||||
|
||||
WEEKLY="$(echo "$PULSE" | grep -o '"weekly_active":[0-9]*' | grep -o '[0-9]*' || echo "0")"
|
||||
CHANGE="$(echo "$PULSE" | grep -o '"change_pct":[0-9-]*' | grep -o '[0-9-]*' || echo "0")"
|
||||
|
||||
echo "Weekly active installs: ${WEEKLY}"
|
||||
if [ "$CHANGE" -gt 0 ] 2>/dev/null; then
|
||||
echo " Change: +${CHANGE}%"
|
||||
elif [ "$CHANGE" -lt 0 ] 2>/dev/null; then
|
||||
echo " Change: ${CHANGE}%"
|
||||
fi
|
||||
echo ""
|
||||
echo "Weekly active installs: ${WEEKLY}"
|
||||
if [ "$CHANGE" -gt 0 ] 2>/dev/null; then
|
||||
echo " Change: +${CHANGE}%"
|
||||
elif [ "$CHANGE" -lt 0 ] 2>/dev/null; then
|
||||
echo " Change: ${CHANGE}%"
|
||||
fi
|
||||
echo ""
|
||||
|
||||
# ─── Skill popularity (top 10) ───────────────────────────────
|
||||
echo "Top skills (last 7 days)"
|
||||
echo "────────────────────────"
|
||||
|
||||
# Query telemetry_events, group by skill
|
||||
EVENTS="$(query "telemetry_events" "select=skill,gstack_version&event_type=eq.skill_run&event_timestamp=gte.${WEEK_AGO}&limit=1000" 2>/dev/null || echo "[]")"
|
||||
|
||||
if [ "$EVENTS" != "[]" ] && [ -n "$EVENTS" ]; then
|
||||
echo "$EVENTS" | grep -o '"skill":"[^"]*"' | awk -F'"' '{print $4}' | sort | uniq -c | sort -rn | head -10 | while read -r COUNT SKILL; do
|
||||
printf " /%-20s %d runs\n" "$SKILL" "$COUNT"
|
||||
# Parse top_skills array from JSON
|
||||
SKILLS="$(echo "$DATA" | grep -o '"top_skills":\[[^]]*\]' || echo "")"
|
||||
if [ -n "$SKILLS" ] && [ "$SKILLS" != '"top_skills":[]' ]; then
|
||||
# Parse each object — handle any key order (JSONB doesn't preserve order)
|
||||
echo "$SKILLS" | grep -o '{[^}]*}' | while read -r OBJ; do
|
||||
SKILL="$(echo "$OBJ" | grep -o '"skill":"[^"]*"' | awk -F'"' '{print $4}')"
|
||||
COUNT="$(echo "$OBJ" | grep -o '"count":[0-9]*' | grep -o '[0-9]*')"
|
||||
[ -n "$SKILL" ] && [ -n "$COUNT" ] && printf " /%-20s %s runs\n" "$SKILL" "$COUNT"
|
||||
done
|
||||
else
|
||||
echo " No data yet"
|
||||
@@ -85,12 +74,12 @@ echo ""
|
||||
echo "Top crash clusters"
|
||||
echo "──────────────────"
|
||||
|
||||
CRASHES="$(query "crash_clusters" "select=error_class,gstack_version,total_occurrences,identified_users&limit=5" 2>/dev/null || echo "[]")"
|
||||
|
||||
if [ "$CRASHES" != "[]" ] && [ -n "$CRASHES" ]; then
|
||||
echo "$CRASHES" | grep -o '"error_class":"[^"]*"' | awk -F'"' '{print $4}' | head -5 | while read -r ERR; do
|
||||
C="$(echo "$CRASHES" | grep -o "\"error_class\":\"$ERR\"[^}]*\"total_occurrences\":[0-9]*" | grep -o '"total_occurrences":[0-9]*' | head -1 | grep -o '[0-9]*')"
|
||||
printf " %-30s %s occurrences\n" "$ERR" "${C:-?}"
|
||||
CRASHES="$(echo "$DATA" | grep -o '"crashes":\[[^]]*\]' || echo "")"
|
||||
if [ -n "$CRASHES" ] && [ "$CRASHES" != '"crashes":[]' ]; then
|
||||
echo "$CRASHES" | grep -o '{[^}]*}' | head -5 | while read -r OBJ; do
|
||||
ERR="$(echo "$OBJ" | grep -o '"error_class":"[^"]*"' | awk -F'"' '{print $4}')"
|
||||
C="$(echo "$OBJ" | grep -o '"total_occurrences":[0-9]*' | grep -o '[0-9]*')"
|
||||
[ -n "$ERR" ] && printf " %-30s %s occurrences\n" "$ERR" "${C:-?}"
|
||||
done
|
||||
else
|
||||
echo " No crashes reported"
|
||||
@@ -101,9 +90,12 @@ echo ""
|
||||
echo "Version distribution (last 7 days)"
|
||||
echo "───────────────────────────────────"
|
||||
|
||||
if [ "$EVENTS" != "[]" ] && [ -n "$EVENTS" ]; then
|
||||
echo "$EVENTS" | grep -o '"gstack_version":"[^"]*"' | awk -F'"' '{print $4}' | sort | uniq -c | sort -rn | head -5 | while read -r COUNT VER; do
|
||||
printf " v%-15s %d events\n" "$VER" "$COUNT"
|
||||
VERSIONS="$(echo "$DATA" | grep -o '"versions":\[[^]]*\]' || echo "")"
|
||||
if [ -n "$VERSIONS" ] && [ "$VERSIONS" != '"versions":[]' ]; then
|
||||
echo "$VERSIONS" | grep -o '{[^}]*}' | head -5 | while read -r OBJ; do
|
||||
VER="$(echo "$OBJ" | grep -o '"version":"[^"]*"' | awk -F'"' '{print $4}')"
|
||||
COUNT="$(echo "$OBJ" | grep -o '"count":[0-9]*' | grep -o '[0-9]*')"
|
||||
[ -n "$VER" ] && [ -n "$COUNT" ] && printf " v%-15s %s events\n" "$VER" "$COUNT"
|
||||
done
|
||||
else
|
||||
echo " No data yet"
|
||||
|
||||
+26
-16
@@ -3,11 +3,12 @@
|
||||
#
|
||||
# Fire-and-forget, backgrounded, rate-limited to once per 5 minutes.
|
||||
# Strips local-only fields before sending. Respects privacy tiers.
|
||||
# Posts to the telemetry-ingest edge function (not PostgREST directly).
|
||||
#
|
||||
# Env overrides (for testing):
|
||||
# GSTACK_STATE_DIR — override ~/.gstack state directory
|
||||
# GSTACK_DIR — override auto-detected gstack root
|
||||
# GSTACK_TELEMETRY_ENDPOINT — override Supabase endpoint URL
|
||||
# GSTACK_SUPABASE_URL — override Supabase project URL
|
||||
set -uo pipefail
|
||||
|
||||
GSTACK_DIR="${GSTACK_DIR:-$(cd "$(dirname "$0")/.." && pwd)}"
|
||||
@@ -19,15 +20,15 @@ RATE_FILE="$ANALYTICS_DIR/.last-sync-time"
|
||||
CONFIG_CMD="$GSTACK_DIR/bin/gstack-config"
|
||||
|
||||
# Source Supabase config if not overridden by env
|
||||
if [ -z "${GSTACK_TELEMETRY_ENDPOINT:-}" ] && [ -f "$GSTACK_DIR/supabase/config.sh" ]; then
|
||||
if [ -z "${GSTACK_SUPABASE_URL:-}" ] && [ -f "$GSTACK_DIR/supabase/config.sh" ]; then
|
||||
. "$GSTACK_DIR/supabase/config.sh"
|
||||
fi
|
||||
ENDPOINT="${GSTACK_TELEMETRY_ENDPOINT:-}"
|
||||
SUPABASE_URL="${GSTACK_SUPABASE_URL:-}"
|
||||
ANON_KEY="${GSTACK_SUPABASE_ANON_KEY:-}"
|
||||
|
||||
# ─── Pre-checks ──────────────────────────────────────────────
|
||||
# No endpoint configured yet → exit silently
|
||||
[ -z "$ENDPOINT" ] && exit 0
|
||||
# No Supabase URL configured yet → exit silently
|
||||
[ -z "$SUPABASE_URL" ] && exit 0
|
||||
|
||||
# No JSONL file → nothing to sync
|
||||
[ -f "$JSONL_FILE" ] || exit 0
|
||||
@@ -66,6 +67,8 @@ UNSENT="$(tail -n "+$SKIP" "$JSONL_FILE" 2>/dev/null || true)"
|
||||
[ -z "$UNSENT" ] && exit 0
|
||||
|
||||
# ─── Strip local-only fields and build batch ─────────────────
|
||||
# Edge function expects raw JSONL field names (v, ts, sessions) —
|
||||
# no column renaming needed (the function maps them internally).
|
||||
BATCH="["
|
||||
FIRST=true
|
||||
COUNT=0
|
||||
@@ -75,13 +78,10 @@ while IFS= read -r LINE; do
|
||||
[ -z "$LINE" ] && continue
|
||||
echo "$LINE" | grep -q '^{' || continue
|
||||
|
||||
# Strip local-only fields + map JSONL field names to Postgres column names
|
||||
# Strip local-only fields (keep v, ts, sessions as-is for edge function)
|
||||
CLEAN="$(echo "$LINE" | sed \
|
||||
-e 's/,"_repo_slug":"[^"]*"//g' \
|
||||
-e 's/,"_branch":"[^"]*"//g' \
|
||||
-e 's/"v":/"schema_version":/g' \
|
||||
-e 's/"ts":/"event_timestamp":/g' \
|
||||
-e 's/"sessions":/"concurrent_sessions":/g' \
|
||||
-e 's/,"repo":"[^"]*"//g')"
|
||||
|
||||
# If anonymous tier, strip installation_id
|
||||
@@ -106,21 +106,31 @@ BATCH="$BATCH]"
|
||||
# Nothing to send after filtering
|
||||
[ "$COUNT" -eq 0 ] && exit 0
|
||||
|
||||
# ─── POST to Supabase ────────────────────────────────────────
|
||||
HTTP_CODE="$(curl -s -o /dev/null -w '%{http_code}' --max-time 10 \
|
||||
-X POST "${ENDPOINT}/telemetry_events" \
|
||||
# ─── POST to edge function ───────────────────────────────────
|
||||
RESP_FILE="$(mktemp /tmp/gstack-sync-XXXXXX 2>/dev/null || echo "/tmp/gstack-sync-$$")"
|
||||
HTTP_CODE="$(curl -s -w '%{http_code}' --max-time 10 \
|
||||
-X POST "${SUPABASE_URL}/functions/v1/telemetry-ingest" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "apikey: ${ANON_KEY}" \
|
||||
-H "Authorization: Bearer ${ANON_KEY}" \
|
||||
-H "Prefer: return=minimal" \
|
||||
-o "$RESP_FILE" \
|
||||
-d "$BATCH" 2>/dev/null || echo "000")"
|
||||
|
||||
# ─── Update cursor on success (2xx) ─────────────────────────
|
||||
case "$HTTP_CODE" in
|
||||
2*) NEW_CURSOR=$(( CURSOR + COUNT ))
|
||||
echo "$NEW_CURSOR" > "$CURSOR_FILE" 2>/dev/null || true ;;
|
||||
2*)
|
||||
# Parse inserted count from response — only advance if events were actually inserted.
|
||||
# Advance by SENT count (not inserted count) because we can't map inserted back to
|
||||
# source lines. If inserted==0, something is systemically wrong — don't advance.
|
||||
INSERTED="$(grep -o '"inserted":[0-9]*' "$RESP_FILE" 2>/dev/null | grep -o '[0-9]*' || echo "0")"
|
||||
if [ "${INSERTED:-0}" -gt 0 ] 2>/dev/null; then
|
||||
NEW_CURSOR=$(( CURSOR + COUNT ))
|
||||
echo "$NEW_CURSOR" > "$CURSOR_FILE" 2>/dev/null || true
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
|
||||
rm -f "$RESP_FILE" 2>/dev/null || true
|
||||
|
||||
# Update rate limit marker
|
||||
touch "$RATE_FILE" 2>/dev/null || true
|
||||
|
||||
|
||||
+7
-10
@@ -160,25 +160,22 @@ fi
|
||||
mkdir -p "$STATE_DIR"
|
||||
|
||||
# Fire Supabase install ping in background (parallel, non-blocking)
|
||||
# This logs an update check event for community health metrics.
|
||||
# If the endpoint isn't configured or Supabase is down, this is a no-op.
|
||||
# Source Supabase config for install ping
|
||||
if [ -z "${GSTACK_TELEMETRY_ENDPOINT:-}" ] && [ -f "$GSTACK_DIR/supabase/config.sh" ]; then
|
||||
# This logs an update check event for community health metrics via edge function.
|
||||
# If Supabase is not configured or telemetry is off, this is a no-op.
|
||||
if [ -z "${GSTACK_SUPABASE_URL:-}" ] && [ -f "$GSTACK_DIR/supabase/config.sh" ]; then
|
||||
. "$GSTACK_DIR/supabase/config.sh"
|
||||
fi
|
||||
_SUPA_ENDPOINT="${GSTACK_TELEMETRY_ENDPOINT:-}"
|
||||
_SUPA_URL="${GSTACK_SUPABASE_URL:-}"
|
||||
_SUPA_KEY="${GSTACK_SUPABASE_ANON_KEY:-}"
|
||||
# Respect telemetry opt-out — don't ping Supabase if user set telemetry: off
|
||||
_TEL_TIER="$("$GSTACK_DIR/bin/gstack-config" get telemetry 2>/dev/null || true)"
|
||||
if [ -n "$_SUPA_ENDPOINT" ] && [ -n "$_SUPA_KEY" ] && [ "${_TEL_TIER:-off}" != "off" ]; then
|
||||
if [ -n "$_SUPA_URL" ] && [ -n "$_SUPA_KEY" ] && [ "${_TEL_TIER:-off}" != "off" ]; then
|
||||
_OS="$(uname -s | tr '[:upper:]' '[:lower:]')"
|
||||
curl -sf --max-time 5 \
|
||||
-X POST "${_SUPA_ENDPOINT}/update_checks" \
|
||||
-X POST "${_SUPA_URL}/functions/v1/update-check" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "apikey: ${_SUPA_KEY}" \
|
||||
-H "Authorization: Bearer ${_SUPA_KEY}" \
|
||||
-H "Prefer: return=minimal" \
|
||||
-d "{\"gstack_version\":\"$LOCAL\",\"os\":\"$_OS\"}" \
|
||||
-d "{\"version\":\"$LOCAL\",\"os\":\"$_OS\"}" \
|
||||
>/dev/null 2>&1 &
|
||||
fi
|
||||
|
||||
|
||||
Reference in New Issue
Block a user