feat: sync heartbeats, eval:trend --team, setup guide, 10 new tests

- 005_sync_heartbeats.sql migration for connectivity testing - eval:trend --team flag pulls team eval data (graceful fallback) - docs/TEAM_SYNC_SETUP.md step-by-step setup guide - Design doc status updated to Phase 2 complete - 10 new tests for sync show formatting functions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-02 11:45:20 +02:00 · 2026-03-15 19:43:03 -05:00
parent 06f2da2019
commit 87cb769c35
5 changed files with 289 additions and 3 deletions
@@ -0,0 +1,132 @@
+# Team Sync Setup Guide
+
+Team sync lets your team share eval results, retro snapshots, QA reports, ship logs, and Greptile triage data via a shared Supabase store. All sync is optional and non-fatal — without it, everything works locally as before.
+
+## Prerequisites
+
+- A [Supabase](https://supabase.com) project (free tier works)
+- gstack v0.3.10+
+
+## Step 1: Create a Supabase project
+
+1. Go to [supabase.com](https://supabase.com) and create a new project
+2. Note your **Project URL** (e.g., `https://xxxx.supabase.co`)
+3. Note your **anon/public key** from Settings > API
+
+## Step 2: Run migrations
+
+In the Supabase SQL Editor, run these files **in order**:
+
+```
+supabase/migrations/001_teams.sql
+supabase/migrations/002_eval_runs.sql
+supabase/migrations/003_data_tables.sql
+supabase/migrations/004_eval_costs.sql
+supabase/migrations/005_sync_heartbeats.sql
+```
+
+Copy-paste each file's contents into the SQL editor and run.
+
+## Step 3: Create your team
+
+In the SQL editor, create a team and add yourself:
+
+```sql
+-- Create team
+INSERT INTO teams (name, slug) VALUES ('Your Team', 'your-team-slug');
+
+-- After authenticating (Step 5), add yourself as owner:
+-- INSERT INTO team_members (team_id, user_id, role)
+-- VALUES ('<team-id>', '<your-user-id>', 'owner');
+```
+
+Note the team slug — you'll need it in the next step.
+
+## Step 4: Configure your project
+
+Copy the example config to your project root:
+
+```bash
+cp .gstack-sync.json.example .gstack-sync.json
+```
+
+Edit `.gstack-sync.json` with your Supabase details:
+
+```json
+{
+  "supabase_url": "https://YOUR_PROJECT.supabase.co",
+  "supabase_anon_key": "eyJ...",
+  "team_slug": "your-team-slug"
+}
+```
+
+**Important:** Add `.gstack-sync.json` to `.gitignore` if it contains sensitive keys, or commit it if your team uses the same Supabase project (the anon key is safe to commit — RLS protects the data).
+
+## Step 5: Authenticate
+
+```bash
+gstack-sync setup
+```
+
+This opens your browser for Supabase OAuth. After authenticating, tokens are saved to `~/.gstack/auth.json` (mode 0600).
+
+**For CI/automation:** Set the `GSTACK_SUPABASE_ACCESS_TOKEN` env var instead of running setup.
+
+## Step 6: Verify
+
+```bash
+gstack-sync test
+```
+
+Expected output:
+```
+gstack sync test
+────────────────────────────────────
+  1. Config:        ok (team: your-team-slug)
+  2. Auth:          ok (you@email.com)
+  3. Push:          ok (123ms)
+  4. Pull:          ok (1 heartbeats, 95ms)
+────────────────────────────────────
+  Sync test passed ✓
+```
+
+## Step 7: See your data
+
+```bash
+gstack-sync show              # team summary dashboard
+gstack-sync show evals        # recent eval runs
+gstack-sync show ships        # recent ship logs
+gstack-sync show retros       # recent retro snapshots
+gstack-sync status            # sync health check
+bun run eval:trend --team     # team-wide test trends
+```
+
+## How it works
+
+When sync is configured, skills automatically push data after completing their primary task:
+
+- `/ship` pushes a ship log after PR creation (Step 8.5)
+- `/retro` pushes the snapshot after saving to `.context/retros/` (Step 13)
+- `/qa` pushes a report after computing the health score (Phase 6)
+- `/review` pushes Greptile triage entries after history file writes
+- Eval runs are pushed automatically by `EvalCollector.finalize()`
+
+All pushes are non-fatal. If sync fails, entries are queued in `~/.gstack/sync-queue.json` and retried on the next push or via `gstack-sync drain`.
+
+## Troubleshooting
+
+| Problem | Fix |
+|---|---|
+| "No .gstack-sync.json found" | Copy `.gstack-sync.json.example` and fill in your values |
+| "Not authenticated" | Run `gstack-sync setup` |
+| Push fails with 404 | Run the migration SQL files in order |
+| "Connection failed" | Check your Supabase URL and that the project is running |
+| Queue growing | Run `gstack-sync drain` to flush |
+
+## Adding team members
+
+Each team member needs to:
+
+1. Have `.gstack-sync.json` in their project (commit it or share it)
+2. Run `gstack-sync setup` to authenticate
+3. Be added to `team_members` in Supabase (by an admin)
@@ -1,7 +1,7 @@
 # Team Coordination Store: gstack as Engineering Intelligence Platform

 > Design doc for the Supabase-backed team data store and universal eval infrastructure.
-> Authored 2026-03-15. Status: approved, not yet implemented.
+> Authored 2026-03-15. Status: Phase 1 complete. Phase 2 complete (skill hooks, sync test/show, team trends). Phase 3-4 not started.

 ## Table of Contents

@@ -541,14 +541,35 @@ async function cmdTrend(args: string[]): Promise<void> {
  let limit = 10;
  let filterTier: string | undefined;
  let filterTest: string | undefined;
+  let useTeam = false;

  for (let i = 0; i < args.length; i++) {
    if (args[i] === '--limit' && args[i + 1]) { limit = parseInt(args[++i], 10); }
    else if (args[i] === '--tier' && args[i + 1]) { filterTier = args[++i]; }
    else if (args[i] === '--test' && args[i + 1]) { filterTest = args[++i]; }
+    else if (args[i] === '--team') { useTeam = true; }
+  }
+
+  let results: EvalResult[];
+  if (useTeam) {
+    try {
+      const { isSyncConfigured } = await import('./sync-config');
+      const { pullEvalRuns } = await import('./sync');
+      if (!isSyncConfigured()) {
+        console.log('Team sync not configured — showing local data only. See docs/TEAM_SYNC_SETUP.md');
+        results = loadEvalResults<EvalResult>(undefined, limit);
+      } else {
+        const teamRows = await pullEvalRuns({ limit });
+        results = teamRows as unknown as EvalResult[];
+      }
+    } catch {
+      console.log('Team sync not available — showing local data only.');
+      results = loadEvalResults<EvalResult>(undefined, limit);
+    }
+  } else {
+    results = loadEvalResults<EvalResult>(undefined, limit);
  }

-  const results = loadEvalResults<EvalResult>(undefined, limit);
  if (results.length === 0) {
    console.log('No eval runs yet. Run: EVALS=1 bun run test:evals');
    return;
@@ -627,7 +648,7 @@ Commands:
  summary [--limit N]                         Aggregate stats across all runs
  push <file>                                 Validate + save + sync an eval result
  cost <file>                                 Show per-model cost breakdown
-  trend [--limit N] [--tier X] [--test X]     Per-test pass rate trends
+  trend [--limit N] [--tier X] [--test X] [--team]  Per-test pass rate trends
  cache read|write|stats|clear|verify         Manage eval cache
  watch                                       Live E2E test dashboard
 `);
@@ -0,0 +1,25 @@
+-- 005_sync_heartbeats.sql — Lightweight table for sync connectivity tests.
+--
+-- Used by `gstack-sync test` to validate the full push/pull flow
+-- without polluting real data tables.
+
+create table if not exists sync_heartbeats (
+  id uuid primary key default gen_random_uuid(),
+  team_id uuid references teams(id) not null,
+  user_id uuid references auth.users(id),
+  hostname text not null default '',
+  timestamp timestamptz not null default now()
+);
+
+-- RLS
+alter table sync_heartbeats enable row level security;
+
+create policy "team_insert" on sync_heartbeats
+  for insert with check (
+    team_id in (select team_id from team_members where user_id = auth.uid())
+  );
+
+create policy "team_read" on sync_heartbeats
+  for select using (
+    team_id in (select team_id from team_members where user_id = auth.uid())
+  );
@@ -0,0 +1,108 @@
+/**
+ * Tests for sync show formatting functions (pure, no network).
+ */
+
+import { describe, test, expect } from 'bun:test';
+import { formatTeamSummary, formatEvalTable, formatShipTable, formatRelativeTime } from '../lib/cli-sync';
+
+describe('formatRelativeTime', () => {
+  test('returns "just now" for recent timestamps', () => {
+    expect(formatRelativeTime(new Date().toISOString())).toBe('just now');
+  });
+
+  test('returns minutes for recent past', () => {
+    const fiveMinAgo = new Date(Date.now() - 5 * 60_000).toISOString();
+    expect(formatRelativeTime(fiveMinAgo)).toBe('5m ago');
+  });
+
+  test('returns hours for older past', () => {
+    const threeHoursAgo = new Date(Date.now() - 3 * 3_600_000).toISOString();
+    expect(formatRelativeTime(threeHoursAgo)).toBe('3h ago');
+  });
+
+  test('returns days for old past', () => {
+    const twoDaysAgo = new Date(Date.now() - 2 * 86_400_000).toISOString();
+    expect(formatRelativeTime(twoDaysAgo)).toBe('2d ago');
+  });
+});
+
+describe('formatTeamSummary', () => {
+  test('formats summary with data', () => {
+    const output = formatTeamSummary({
+      teamSlug: 'test-team',
+      evalRuns: [
+        { timestamp: new Date().toISOString(), user_id: 'u1', tests: [{ detection_rate: 4 }] },
+        { timestamp: new Date().toISOString(), user_id: 'u2', tests: [{ detection_rate: 5 }] },
+      ],
+      shipLogs: [
+        { created_at: new Date().toISOString() },
+      ],
+      retroSnapshots: [
+        { date: '2026-03-15', streak_days: 47 },
+      ],
+      queueSize: 0,
+      cacheLastPull: new Date().toISOString(),
+    });
+
+    expect(output).toContain('test-team');
+    expect(output).toContain('2 runs');
+    expect(output).toContain('2 contributors');
+    expect(output).toContain('1 PRs');
+    expect(output).toContain('4.5');  // avg detection
+    expect(output).toContain('streak: 47d');
+    expect(output).toContain('0 items');
+  });
+
+  test('handles empty data gracefully', () => {
+    const output = formatTeamSummary({
+      teamSlug: 'empty-team',
+      evalRuns: [],
+      shipLogs: [],
+      retroSnapshots: [],
+      queueSize: 3,
+      cacheLastPull: null,
+    });
+
+    expect(output).toContain('empty-team');
+    expect(output).toContain('0 runs');
+    expect(output).toContain('0 PRs');
+    expect(output).toContain('3 items');
+    expect(output).toContain('never');
+  });
+});
+
+describe('formatEvalTable', () => {
+  test('formats eval runs as table', () => {
+    const output = formatEvalTable([
+      { timestamp: '2026-03-15T12:00:00Z', branch: 'main', passed: 10, total_tests: 10, total_cost_usd: 2.50, tier: 'e2e' },
+    ]);
+
+    expect(output).toContain('Recent Eval Runs');
+    expect(output).toContain('2026-03-15');
+    expect(output).toContain('main');
+    expect(output).toContain('10/10');
+    expect(output).toContain('$2.50');
+    expect(output).toContain('e2e');
+  });
+
+  test('returns message for empty data', () => {
+    expect(formatEvalTable([])).toContain('No eval runs yet');
+  });
+});
+
+describe('formatShipTable', () => {
+  test('formats ship logs as table', () => {
+    const output = formatShipTable([
+      { created_at: '2026-03-15T12:00:00Z', version: '0.3.10', branch: 'feature/sync', pr_url: 'https://github.com/org/repo/pull/1' },
+    ]);
+
+    expect(output).toContain('Recent Ship Logs');
+    expect(output).toContain('0.3.10');
+    expect(output).toContain('feature/sync');
+    expect(output).toContain('github.com');
+  });
+
+  test('returns message for empty data', () => {
+    expect(formatShipTable([])).toContain('No ship logs yet');
+  });
+});