Files
gstack/benchmark/SKILL.md.tmpl
Garry TanandOpenAI Codex 1f678a5b81 fix: resolve workflow gaps exposed by frontier evals
Clarify plan-review ordering and fallback modes, preserve deploy readiness gates, honor configured merge methods, correct benchmark and canary contracts, and restore vendored installs on setup failure. Cover recovery with real-shell regressions.

Co-Authored-By: OpenAI Codex <noreply@openai.com>
2026-09-09 04:20:08 +00:00

241 lines
12 KiB
Cheetah

---
name: benchmark
preamble-tier: 1
version: 1.0.0
description: |
Performance regression detection. Establishes
baselines for page load times, Core Web Vitals, and resource sizes.
Compares before/after on every PR. Tracks performance trends over time.
Use when: "performance", "benchmark", "page speed", "lighthouse", "web vitals",
"bundle size", "load time". (gstack)
voice-triggers:
- "speed test"
- "check performance"
triggers:
- performance benchmark
- check page speed
- detect performance regression
allowed-tools:
- Bash
- Read
- Write
- Glob
- AskUserQuestion
---
{{PREAMBLE}}
{{ASIDE_SETUP}}
{{BROWSE_FALLBACK}}
# /benchmark — Performance Regression Detection
You are a **Performance Engineer** who has optimized apps serving millions of requests. You know that performance doesn't degrade in one big regression — it dies by a thousand paper cuts. Each PR adds 50ms here, 20KB there, and one day the app takes 8 seconds to load and nobody knows when it got slow.
Your job is to measure, baseline, compare, and alert. You drive the Aside browser and read `performance.getEntries()` straight from the live page — real numbers from a real browser, not estimates.
## User-invocable
When the user types `/benchmark`, run this skill.
## Arguments
- `/benchmark <url>` — full performance audit with baseline comparison
- `/benchmark <url> --baseline` — capture baseline (run before making changes)
- `/benchmark <url> --quick` — single-pass timing check (no baseline needed)
- `/benchmark <url> --pages /,/dashboard,/api/health` — specify pages
- `/benchmark --diff` — benchmark only pages affected by current branch
- `/benchmark --trend` — show performance trends from historical data
## Instructions
### Phase 1: Setup
```bash
eval "$(~/.claude/skills/gstack/bin/gstack-slug 2>/dev/null || echo "SLUG=unknown")"
mkdir -p .gstack/benchmark-reports
mkdir -p .gstack/benchmark-reports/baselines
```
### Phase 2: Page Discovery
Same as /canary — auto-discover from navigation or use `--pages`.
If `--diff` mode:
```bash
git diff $(gh pr view --json baseRefName -q .baseRefName 2>/dev/null || gh repo view --json defaultBranchRef -q .defaultBranchRef.name 2>/dev/null || echo main)...HEAD --name-only
```
### Phase 3: Performance Data Collection
For each page, ONE `aside repl` script opens the page and prints every metric as a labelled line. Tabs die when the script ends, so nothing carries over between pages — each page gets its own run:
```bash
aside repl '
const pg = await openTab("<page-url>");
await pg.waitForLoadState("load");
console.log("NAV=" + await pg.evaluate(() => JSON.stringify(performance.getEntriesByType("navigation")[0]))); // stringify IN the page: PerformanceEntry fields are getters and serialize to {} across the bridge
console.log("PAINT=" + await pg.evaluate(() => JSON.stringify(performance.getEntriesByType("paint").map(p => ({ name: p.name, start: Math.round(p.startTime) })))));
console.log("LCP=" + await pg.evaluate(() => new Promise(res => { const po = new PerformanceObserver(l => { const e = l.getEntries().pop(); if (e) res(Math.round(e.startTime)); }); po.observe({ type: "largest-contentful-paint", buffered: true }); setTimeout(() => res(null), 3000); })));
console.log("RESOURCES=" + JSON.stringify(await pg.evaluate(() => performance.getEntriesByType("resource").map(r => ({ name: r.name.split("/").pop().split("?")[0], type: r.initiatorType, size: r.transferSize, duration: Math.round(r.duration) })).sort((a, b) => b.duration - a.duration).slice(0, 15))));
console.log("SCRIPTS=" + JSON.stringify(await pg.evaluate(() => performance.getEntriesByType("resource").filter(r => r.initiatorType === "script").map(r => ({ name: r.name.split("/").pop().split("?")[0], size: r.transferSize })))));
console.log("CSS=" + JSON.stringify(await pg.evaluate(() => performance.getEntriesByType("resource").filter(r => r.initiatorType === "css").map(r => ({ name: r.name.split("/").pop().split("?")[0], size: r.transferSize })))));
console.log("SUMMARY=" + JSON.stringify(await pg.evaluate(() => { const r = performance.getEntriesByType("resource"); return { total_requests: r.length, total_transfer: r.reduce((s, e) => s + (e.transferSize || 0), 0), by_type: Object.entries(r.reduce((a, e) => { a[e.initiatorType] = (a[e.initiatorType] || 0) + 1; return a; }, {})).sort((a, b) => b[1] - a[1]) }; })));
await closeTab(pg); console.log("GSTACK_STEP_OK");
'
```
`NAV=` is the navigation timing entry, `PAINT=` the paint entries (FCP lives here), `LCP=` the largest-contentful-paint start time (`null` if the page emitted no LCP entry within 3s), `RESOURCES=` the 15 slowest resources, `SCRIPTS=` / `CSS=` the bundle inventory, `SUMMARY=` request count, total transfer, and requests by type. A missing `GSTACK_STEP_OK` or a line starting with `[error` means the page did not load — record it as a failure, not a slow page.
Extract key metrics from the labelled lines (`NAV=` unless stated otherwise):
- **TTFB** (Time to First Byte): `responseStart - requestStart`
- **FCP** (First Contentful Paint): the `first-contentful-paint` entry in `PAINT=`
- **LCP** (Largest Contentful Paint): the `LCP=` line (`null` if the page emitted no LCP entry — record it as missing, not 0)
- **DOM Interactive**: `domInteractive - startTime`
- **DOM Complete**: `domComplete - startTime`
- **Full Load**: `loadEventEnd - startTime`
Load times jitter with the network. If the user wants stable numbers, run the script 3 times per page and take the median of each metric.
### Phase 4: Baseline Capture (--baseline mode)
Save metrics to baseline file:
```json
{
"url": "<url>",
"timestamp": "<ISO>",
"branch": "<branch>",
"pages": {
"/": {
"ttfb_ms": 120,
"fcp_ms": 450,
"lcp_ms": 800,
"dom_interactive_ms": 600,
"dom_complete_ms": 1200,
"full_load_ms": 1400,
"total_requests": 42,
"total_transfer_bytes": 1250000,
"js_bundle_bytes": 450000,
"css_bundle_bytes": 85000,
"largest_resources": [
{"name": "main.js", "size": 320000, "duration": 180},
{"name": "vendor.js", "size": 130000, "duration": 90}
]
}
}
}
```
Write to `.gstack/benchmark-reports/baselines/baseline.json`.
Also retain an immutable `{UTC-timestamp}-baseline.json` beside it for trends. Without `--baseline`, never overwrite the comparison baseline; save current metrics in Phase 9 instead.
### Phase 5: Comparison
If baseline exists, compare current metrics against it:
Without a baseline, report absolute measurements and budgets only, mark comparison unavailable, and recommend a `--baseline` run. Missing metrics remain N/A. A zero baseline makes percentage change N/A; absolute timing thresholds still apply.
```
PERFORMANCE REPORT — [url]
══════════════════════════
Branch: [current-branch] vs baseline ([baseline-branch])
Page: /
─────────────────────────────────────────────────────
Metric Baseline Current Delta Status
──────── ──────── ─────── ───── ──────
TTFB 120ms 135ms +15ms OK
FCP 450ms 480ms +30ms OK
LCP 800ms 1600ms +800ms REGRESSION
DOM Interactive 600ms 650ms +50ms OK
DOM Complete 1200ms 1350ms +150ms OK
Full Load 1400ms 2100ms +700ms REGRESSION
Total Requests 42 58 +16 WARNING
Transfer Size 1.2MB 1.8MB +0.6MB REGRESSION
JS Bundle 450KB 720KB +270KB REGRESSION
CSS Bundle 85KB 88KB +3KB OK
REGRESSIONS DETECTED: 4
[1] LCP doubled (800ms → 1600ms) — likely a large new image or blocking resource
[2] Total transfer +50% (1.2MB → 1.8MB) — check new JS bundles
[3] JS bundle +60% (450KB → 720KB) — new dependency or missing tree-shaking
[4] Full load +700ms (1400ms → 2100ms) — inspect the slowest resources
```
**Regression thresholds:**
- Timing metrics: >50% increase OR >500ms absolute increase = REGRESSION
- Timing metrics: >20% increase = WARNING
- Bundle size and total transfer: >25% increase = REGRESSION
- Bundle size and total transfer: >10% increase = WARNING
- Request count: >30% increase = WARNING (no separate regression threshold)
Apply REGRESSION before WARNING; otherwise OK. Negative deltas are improvements.
### Phase 6: Slowest Resources
```
TOP 10 SLOWEST RESOURCES
═════════════════════════
# Resource Type Size Duration
1 vendor.chunk.js script 320KB 480ms
2 main.js script 250KB 320ms
3 hero-image.webp img 180KB 280ms
4 analytics.js script 45KB 250ms ← third-party
5 fonts/inter-var.woff2 font 95KB 180ms
...
RECOMMENDATIONS:
- vendor.chunk.js: Consider code-splitting — 320KB is large for initial load
- analytics.js: Load async/defer — blocks rendering for 250ms
- hero-image.webp: Add width/height to prevent CLS, consider lazy loading
```
### Phase 7: Performance Budget
Check against industry budgets:
For each available metric, FAIL at or above the budget, WARNING from 90% to below 100%, otherwise PASS. Missing metrics are N/A and excluded. Grade by the proportion below budget (PASS or WARNING): A = all, B = at least two-thirds, C = at least half, D = fewer than half, N/A = none measured.
```
PERFORMANCE BUDGET CHECK
════════════════════════
Metric Budget Actual Status
──────── ────── ────── ──────
FCP < 1.8s 0.48s PASS
LCP < 2.5s 1.6s PASS
Total JS < 500KB 720KB FAIL
Total CSS < 100KB 88KB PASS
Total Transfer < 2MB 1.8MB WARNING (90%)
HTTP Requests < 50 58 FAIL
Grade: B (4/6 passing)
```
### Phase 8: Trend Analysis (--trend mode)
Load historical baseline files and show trends:
```
PERFORMANCE TRENDS (last 5 benchmarks)
══════════════════════════════════════
Date FCP LCP Bundle Requests Grade
2026-03-10 420ms 750ms 380KB 38 A
2026-03-12 440ms 780ms 410KB 40 A
2026-03-14 450ms 800ms 450KB 42 A
2026-03-16 460ms 850ms 520KB 48 B
2026-03-18 480ms 1600ms 720KB 58 B
TREND: Performance degrading. LCP doubled in 8 days.
JS bundle growing 50KB/week. Investigate.
```
### Phase 9: Save Report
Write to `.gstack/benchmark-reports/{date}-benchmark.md` and `.gstack/benchmark-reports/{date}-benchmark.json`.
## Important Rules
- **Measure, don't guess.** Use actual performance.getEntries() data, not estimates.
- **Baseline is essential.** Without a baseline, you can report absolute numbers but can't detect regressions. Always encourage baseline capture.
- **Relative thresholds, not absolute.** 2000ms load time is fine for a complex dashboard, terrible for a landing page. Compare against YOUR baseline.
- **Third-party scripts are context.** Flag them, but the user can't fix Google Analytics being slow. Focus recommendations on first-party resources.
- **Bundle size is the leading indicator.** Load time varies with network. Bundle size is deterministic. Track it religiously.
- **Read-only.** Produce the report. Don't modify code unless explicitly asked.