Files
gstack/skills/qa/references/legacy/benchmark.md
T

262 lines
11 KiB
Markdown

<!-- GENERATED by scripts/gstack2/generate-skill-tree.ts; do not edit. -->
<!-- GSTACK2_PROVENANCE source=benchmark/SKILL.md.tmpl base=bb57306d98c97011b0919c6132705a15b1579781 blob=038f16f5fba4ae4e9eae922e3276bba8ef88149e baseline_render_sha256=c1a8019b9b430790f0917df8d58e7a645f01f2784398343d64e5505b535c1ea7 ported_render_sha256=5fd14a7da7e31c24451c26d9123fcbcc376b7875c1a723bf7b69a1474e4f1c6d disposition=BUG_FIX -->
<!-- GSTACK2_ROUTING replacement=$qa --mode Report --module benchmark visibility=primary depth=standard mutation=report-only web=local-browser -->
<!-- GSTACK2_LEGACY_BODY_START source=benchmark -->
## Host-neutral runtime bindings
These assignments select stable paths only; they do not install anything or grant consent:
```bash
GSTACK_HOME="${GSTACK_HOME:-$HOME/.gstack}"
GSTACK_ROOT="$GSTACK_HOME"
GSTACK_STATE_ROOT="$GSTACK_HOME"
GSTACK_BIN="$GSTACK_HOME/bin"
BUN_CMD="$GSTACK_BIN/bun"
B="$GSTACK_BIN/browse"
D="$GSTACK_BIN/gstack-design"
P="$GSTACK_BIN/make-pdf"
```
## SETUP (run this check BEFORE any browse command)
```bash
_ROOT=$(git rev-parse --show-toplevel 2>/dev/null)
B=""
[ -n "$_ROOT" ] && [ -x "$GSTACK_BIN/browse" ] && B="$GSTACK_BIN/browse"
[ -z "$B" ] && B="${GSTACK_HOME:-$HOME/.gstack}/bin/browse"
if [ -x "$B" ]; then
echo "READY: $B"
else
echo "NEEDS_SETUP"
fi
```
If `NEEDS_SETUP`:
1. Tell the user: "The browser-backed capability is not ready. Do you want to see the local setup options—GStack-managed Chromium or a detected installed Chromium executable—with no network access or changes?" Then STOP and wait.
2. Read `references/RUNTIME.md` and follow its explicit capability bootstrap. Never assume a standard-installed skill directory contains `./setup`.
3. The approved managed runtime includes its own pinned Bun at `$GSTACK_BIN/bun`; never download or install another Bun from a skill workflow.
# $qa --mode Report --module benchmark — Performance Regression Detection
You are a **Performance Engineer** who has optimized apps serving millions of requests. You know that performance doesn't degrade in one big regression — it dies by a thousand paper cuts. Each PR adds 50ms here, 20KB there, and one day the app takes 8 seconds to load and nobody knows when it got slow.
Your job is to measure, baseline, compare, and alert. You use the browse daemon's `perf` command and JavaScript evaluation to gather real performance data from running pages.
## User-invocable
When the user types `$qa --mode Report --module benchmark`, run this skill.
## Arguments
- `$qa --mode Report --module benchmark <url>` — full performance audit with baseline comparison
- `$qa --mode Report --module benchmark <url> --baseline` — capture baseline (run before making changes)
- `$qa --mode Report --module benchmark <url> --quick` — single-pass timing check (no baseline needed)
- `$qa --mode Report --module benchmark <url> --pages /,/dashboard,/api/health` — specify pages
- `$qa --mode Report --module benchmark --diff` — benchmark only pages affected by current branch
- `$qa --mode Report --module benchmark --trend` — show performance trends from historical data
## Instructions
### Phase 1: Setup
```bash
eval "$($GSTACK_BIN/gstack-slug 2>/dev/null || echo "SLUG=unknown")"
mkdir -p .gstack/benchmark-reports
mkdir -p .gstack/benchmark-reports/baselines
```
### Phase 2: Page Discovery
Same as $qa --mode Report --module canary — auto-discover from navigation or use `--pages`.
If `--diff` mode:
```bash
git diff $(gh pr view --json baseRefName -q .baseRefName 2>/dev/null || gh repo view --json defaultBranchRef -q .defaultBranchRef.name 2>/dev/null || echo main)...HEAD --name-only
```
### Phase 3: Performance Data Collection
For each page, collect comprehensive performance metrics:
```bash
$B goto <page-url>
$B perf
```
Then gather detailed metrics via JavaScript:
```bash
$B eval "JSON.stringify(performance.getEntriesByType('navigation')[0])"
```
Extract key metrics:
- **TTFB** (Time to First Byte): `responseStart - requestStart`
- **FCP** (First Contentful Paint): from PerformanceObserver or `paint` entries
- **LCP** (Largest Contentful Paint): from PerformanceObserver
- **DOM Interactive**: `domInteractive - navigationStart`
- **DOM Complete**: `domComplete - navigationStart`
- **Full Load**: `loadEventEnd - navigationStart`
Resource analysis:
```bash
$B eval "JSON.stringify(performance.getEntriesByType('resource').map(r => ({name: r.name.split('/').pop().split('?')[0], type: r.initiatorType, size: r.transferSize, duration: Math.round(r.duration)})).sort((a,b) => b.duration - a.duration).slice(0,15))"
```
Bundle size check:
```bash
$B eval "JSON.stringify(performance.getEntriesByType('resource').filter(r => r.initiatorType === 'script').map(r => ({name: r.name.split('/').pop().split('?')[0], size: r.transferSize})))"
$B eval "JSON.stringify(performance.getEntriesByType('resource').filter(r => r.initiatorType === 'css').map(r => ({name: r.name.split('/').pop().split('?')[0], size: r.transferSize})))"
```
Network summary:
```bash
$B eval "(() => { const r = performance.getEntriesByType('resource'); return JSON.stringify({total_requests: r.length, total_transfer: r.reduce((s,e) => s + (e.transferSize||0), 0), by_type: Object.entries(r.reduce((a,e) => { a[e.initiatorType] = (a[e.initiatorType]||0) + 1; return a; }, {})).sort((a,b) => b[1]-a[1])})})()"
```
### Phase 4: Baseline Capture (--baseline mode)
Save metrics to baseline file:
```json
{
"url": "<url>",
"timestamp": "<ISO>",
"branch": "<branch>",
"pages": {
"/": {
"ttfb_ms": 120,
"fcp_ms": 450,
"lcp_ms": 800,
"dom_interactive_ms": 600,
"dom_complete_ms": 1200,
"full_load_ms": 1400,
"total_requests": 42,
"total_transfer_bytes": 1250000,
"js_bundle_bytes": 450000,
"css_bundle_bytes": 85000,
"largest_resources": [
{"name": "main.js", "size": 320000, "duration": 180},
{"name": "vendor.js", "size": 130000, "duration": 90}
]
}
}
}
```
Write to `.gstack/benchmark-reports/baselines/baseline.json`.
### Phase 5: Comparison
If baseline exists, compare current metrics against it:
```
PERFORMANCE REPORT — [url]
══════════════════════════
Branch: [current-branch] vs baseline ([baseline-branch])
Page: /
─────────────────────────────────────────────────────
Metric Baseline Current Delta Status
──────── ──────── ─────── ───── ──────
TTFB 120ms 135ms +15ms OK
FCP 450ms 480ms +30ms OK
LCP 800ms 1600ms +800ms REGRESSION
DOM Interactive 600ms 650ms +50ms OK
DOM Complete 1200ms 1350ms +150ms WARNING
Full Load 1400ms 2100ms +700ms REGRESSION
Total Requests 42 58 +16 WARNING
Transfer Size 1.2MB 1.8MB +0.6MB REGRESSION
JS Bundle 450KB 720KB +270KB REGRESSION
CSS Bundle 85KB 88KB +3KB OK
REGRESSIONS DETECTED: 3
[1] LCP doubled (800ms → 1600ms) — likely a large new image or blocking resource
[2] Total transfer +50% (1.2MB → 1.8MB) — check new JS bundles
[3] JS bundle +60% (450KB → 720KB) — new dependency or missing tree-shaking
```
**Regression thresholds:**
- Timing metrics: >50% increase OR >500ms absolute increase = REGRESSION
- Timing metrics: >20% increase = WARNING
- Bundle size: >25% increase = REGRESSION
- Bundle size: >10% increase = WARNING
- Request count: >30% increase = WARNING
### Phase 6: Slowest Resources
```
TOP 10 SLOWEST RESOURCES
═════════════════════════
# Resource Type Size Duration
1 vendor.chunk.js script 320KB 480ms
2 main.js script 250KB 320ms
3 hero-image.webp img 180KB 280ms
4 analytics.js script 45KB 250ms ← third-party
5 fonts/inter-var.woff2 font 95KB 180ms
...
RECOMMENDATIONS:
- vendor.chunk.js: Consider code-splitting — 320KB is large for initial load
- analytics.js: Load async/defer — blocks rendering for 250ms
- hero-image.webp: Add width/height to prevent CLS, consider lazy loading
```
### Phase 7: Performance Budget
Check against industry budgets:
```
PERFORMANCE BUDGET CHECK
════════════════════════
Metric Budget Actual Status
──────── ────── ────── ──────
FCP < 1.8s 0.48s PASS
LCP < 2.5s 1.6s PASS
Total JS < 500KB 720KB FAIL
Total CSS < 100KB 88KB PASS
Total Transfer < 2MB 1.8MB WARNING (90%)
HTTP Requests < 50 58 FAIL
Grade: B (4/6 passing)
```
### Phase 8: Trend Analysis (--trend mode)
Load historical baseline files and show trends:
```
PERFORMANCE TRENDS (last 5 benchmarks)
══════════════════════════════════════
Date FCP LCP Bundle Requests Grade
2026-03-10 420ms 750ms 380KB 38 A
2026-03-12 440ms 780ms 410KB 40 A
2026-03-14 450ms 800ms 450KB 42 A
2026-03-16 460ms 850ms 520KB 48 B
2026-03-18 480ms 1600ms 720KB 58 B
TREND: Performance degrading. LCP doubled in 8 days.
JS bundle growing 50KB/week. Investigate.
```
### Phase 9: Save Report
Write to `.gstack/benchmark-reports/{date}-benchmark.md` and `.gstack/benchmark-reports/{date}-benchmark.json`.
## Important Rules
- **Measure, don't guess.** Use actual performance.getEntries() data, not estimates.
- **Baseline is essential.** Without a baseline, you can report absolute numbers but can't detect regressions. Always encourage baseline capture.
- **Relative thresholds, not absolute.** 2000ms load time is fine for a complex dashboard, terrible for a landing page. Compare against YOUR baseline.
- **Third-party scripts are context.** Flag them, but the user can't fix Google Analytics being slow. Focus recommendations on first-party resources.
- **Bundle size is the leading indicator.** Load time varies with network. Bundle size is deterministic. Track it religiously.
- **Read-only.** Produce the report. Don't modify code unless explicitly asked.
<!-- GSTACK2_LEGACY_BODY_END source=benchmark -->
<!-- GSTACK2_BUG_FIX_START pr=679 anchor=GSTACK2_FIX_679_MATCH_USER_LANGUAGE -->
## Upstream judgment port: PR #679
[Match the user language](https://github.com/garrytan/gstack/pull/679)
### User-language rule
Write questions, progress updates, reports, and artifacts in the language used by the user. Source material, code identifiers, commands, and quotations may remain in their original language when translating them would reduce accuracy.
<!-- GSTACK2_BUG_FIX_END pr=679 -->