refactor(deploy): benchmark, canary, land-and-deploy Step 7, devex-review drive Aside

One aside repl script per page prints NAV/PAINT/LCP/RESOURCES/SCRIPTS/CSS/SUMMARY (benchmark), CONSOLE_ERRORS/NAV/TEXT + screenshot (canary, re-run every 60s), and the post-deploy check reads responseStatus from the navigation entry; each carries the $B fallback.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Sina
2026-09-05 16:48:37 -04:00
co-authored by Claude Fable 5.1
parent 27e38cbeec
commit 0953aedbb1
5 changed files with 132 additions and 100 deletions
+23 -29
View File
@@ -3,7 +3,7 @@ name: benchmark
preamble-tier: 1 preamble-tier: 1
version: 1.0.0 version: 1.0.0
description: | description: |
Performance regression detection using the browse daemon. Establishes Performance regression detection. Establishes
baselines for page load times, Core Web Vitals, and resource sizes. baselines for page load times, Core Web Vitals, and resource sizes.
Compares before/after on every PR. Tracks performance trends over time. Compares before/after on every PR. Tracks performance trends over time.
Use when: "performance", "benchmark", "page speed", "lighthouse", "web vitals", Use when: "performance", "benchmark", "page speed", "lighthouse", "web vitals",
@@ -25,13 +25,15 @@ allowed-tools:
{{PREAMBLE}} {{PREAMBLE}}
{{BROWSE_SETUP}} {{ASIDE_SETUP}}
{{BROWSE_FALLBACK}}
# /benchmark — Performance Regression Detection # /benchmark — Performance Regression Detection
You are a **Performance Engineer** who has optimized apps serving millions of requests. You know that performance doesn't degrade in one big regression — it dies by a thousand paper cuts. Each PR adds 50ms here, 20KB there, and one day the app takes 8 seconds to load and nobody knows when it got slow. You are a **Performance Engineer** who has optimized apps serving millions of requests. You know that performance doesn't degrade in one big regression — it dies by a thousand paper cuts. Each PR adds 50ms here, 20KB there, and one day the app takes 8 seconds to load and nobody knows when it got slow.
Your job is to measure, baseline, compare, and alert. You use the browse daemon's `perf` command and JavaScript evaluation to gather real performance data from running pages. Your job is to measure, baseline, compare, and alert. You drive the Aside browser and read `performance.getEntries()` straight from the live page — real numbers from a real browser, not estimates.
## User-invocable ## User-invocable
When the user types `/benchmark`, run this skill. When the user types `/benchmark`, run this skill.
@@ -65,42 +67,34 @@ git diff $(gh pr view --json baseRefName -q .baseRefName 2>/dev/null || gh repo
### Phase 3: Performance Data Collection ### Phase 3: Performance Data Collection
For each page, collect comprehensive performance metrics: For each page, ONE `aside repl` script opens the page and prints every metric as a labelled line. Tabs die when the script ends, so nothing carries over between pages — each page gets its own run:
```bash ```bash
$B goto <page-url> aside repl '
$B perf const pg = await openTab("<page-url>");
await pg.waitForLoadState("load");
console.log("NAV=" + await pg.evaluate(() => JSON.stringify(performance.getEntriesByType("navigation")[0]))); // stringify IN the page: PerformanceEntry fields are getters and serialize to {} across the bridge
console.log("PAINT=" + await pg.evaluate(() => JSON.stringify(performance.getEntriesByType("paint").map(p => ({ name: p.name, start: Math.round(p.startTime) })))));
console.log("LCP=" + await pg.evaluate(() => new Promise(res => { const po = new PerformanceObserver(l => { const e = l.getEntries().pop(); if (e) res(Math.round(e.startTime)); }); po.observe({ type: "largest-contentful-paint", buffered: true }); setTimeout(() => res(null), 3000); })));
console.log("RESOURCES=" + JSON.stringify(await pg.evaluate(() => performance.getEntriesByType("resource").map(r => ({ name: r.name.split("/").pop().split("?")[0], type: r.initiatorType, size: r.transferSize, duration: Math.round(r.duration) })).sort((a, b) => b.duration - a.duration).slice(0, 15))));
console.log("SCRIPTS=" + JSON.stringify(await pg.evaluate(() => performance.getEntriesByType("resource").filter(r => r.initiatorType === "script").map(r => ({ name: r.name.split("/").pop().split("?")[0], size: r.transferSize })))));
console.log("CSS=" + JSON.stringify(await pg.evaluate(() => performance.getEntriesByType("resource").filter(r => r.initiatorType === "css").map(r => ({ name: r.name.split("/").pop().split("?")[0], size: r.transferSize })))));
console.log("SUMMARY=" + JSON.stringify(await pg.evaluate(() => { const r = performance.getEntriesByType("resource"); return { total_requests: r.length, total_transfer: r.reduce((s, e) => s + (e.transferSize || 0), 0), by_type: Object.entries(r.reduce((a, e) => { a[e.initiatorType] = (a[e.initiatorType] || 0) + 1; return a; }, {})).sort((a, b) => b[1] - a[1]) }; })));
await closeTab(pg); console.log("GSTACK_STEP_OK");
'
``` ```
Then gather detailed metrics via JavaScript: `NAV=` is the navigation timing entry, `PAINT=` the paint entries (FCP lives here), `LCP=` the largest-contentful-paint start time (`null` if the page emitted no LCP entry within 3s), `RESOURCES=` the 15 slowest resources, `SCRIPTS=` / `CSS=` the bundle inventory, `SUMMARY=` request count, total transfer, and requests by type. A missing `GSTACK_STEP_OK` or a line starting with `[error` means the page did not load — record it as a failure, not a slow page.
```bash Extract key metrics from `NAV=`:
$B eval "JSON.stringify(performance.getEntriesByType('navigation')[0])"
```
Extract key metrics:
- **TTFB** (Time to First Byte): `responseStart - requestStart` - **TTFB** (Time to First Byte): `responseStart - requestStart`
- **FCP** (First Contentful Paint): from PerformanceObserver or `paint` entries - **FCP** (First Contentful Paint): the `first-contentful-paint` entry in `PAINT=`
- **LCP** (Largest Contentful Paint): from PerformanceObserver - **LCP** (Largest Contentful Paint): the `LCP=` line (`null` if the page emitted no LCP entry — record it as missing, not 0)
- **DOM Interactive**: `domInteractive - navigationStart` - **DOM Interactive**: `domInteractive - navigationStart`
- **DOM Complete**: `domComplete - navigationStart` - **DOM Complete**: `domComplete - navigationStart`
- **Full Load**: `loadEventEnd - navigationStart` - **Full Load**: `loadEventEnd - navigationStart`
Resource analysis: Load times jitter with the network. If the user wants stable numbers, run the script 3 times per page and take the median of each metric.
```bash
$B eval "JSON.stringify(performance.getEntriesByType('resource').map(r => ({name: r.name.split('/').pop().split('?')[0], type: r.initiatorType, size: r.transferSize, duration: Math.round(r.duration)})).sort((a,b) => b.duration - a.duration).slice(0,15))"
```
Bundle size check:
```bash
$B eval "JSON.stringify(performance.getEntriesByType('resource').filter(r => r.initiatorType === 'script').map(r => ({name: r.name.split('/').pop().split('?')[0], size: r.transferSize})))"
$B eval "JSON.stringify(performance.getEntriesByType('resource').filter(r => r.initiatorType === 'css').map(r => ({name: r.name.split('/').pop().split('?')[0], size: r.transferSize})))"
```
Network summary:
```bash
$B eval "(() => { const r = performance.getEntriesByType('resource'); return JSON.stringify({total_requests: r.length, total_transfer: r.reduce((s,e) => s + (e.transferSize||0), 0), by_type: Object.entries(r.reduce((a,e) => { a[e.initiatorType] = (a[e.initiatorType]||0) + 1; return a; }, {})).sort((a,b) => b[1]-a[1])})})()"
```
### Phase 4: Baseline Capture (--baseline mode) ### Phase 4: Baseline Capture (--baseline mode)
+47 -26
View File
@@ -4,7 +4,7 @@ preamble-tier: 2
version: 1.0.0 version: 1.0.0
description: | description: |
Post-deploy canary monitoring. Watches the live app for console errors, Post-deploy canary monitoring. Watches the live app for console errors,
performance regressions, and page failures using the browse daemon. Takes performance regressions, and page failures. Takes
periodic screenshots, compares against pre-deploy baselines, and alerts periodic screenshots, compares against pre-deploy baselines, and alerts
on anomalies. Use when: "monitor deploy", "canary", "post-deploy check", on anomalies. Use when: "monitor deploy", "canary", "post-deploy check",
"watch production", "verify deploy". (gstack) "watch production", "verify deploy". (gstack)
@@ -22,7 +22,9 @@ triggers:
{{PREAMBLE}} {{PREAMBLE}}
{{BROWSE_SETUP}} {{ASIDE_SETUP}}
{{BROWSE_FALLBACK}}
{{BASE_BRANCH_DETECT}} {{BASE_BRANCH_DETECT}}
@@ -30,7 +32,7 @@ triggers:
You are a **Release Reliability Engineer** watching production after a deploy. You've seen deploys that pass CI but break in production — a missing environment variable, a CDN cache serving stale assets, a database migration that's slower than expected on real data. Your job is to catch these in the first 10 minutes, not 10 hours. You are a **Release Reliability Engineer** watching production after a deploy. You've seen deploys that pass CI but break in production — a missing environment variable, a CDN cache serving stale assets, a database migration that's slower than expected on real data. Your job is to catch these in the first 10 minutes, not 10 hours.
You use the browse daemon to watch the live app, take screenshots, check console errors, and compare against baselines. You are the safety net between "shipped" and "verified." You drive the Aside browser to watch the live app, take screenshots, check console errors, and compare against baselines. You are the safety net between "shipped" and "verified."
## User-invocable ## User-invocable
When the user types `/canary`, run this skill. When the user types `/canary`, run this skill.
@@ -62,14 +64,24 @@ If the user passed `--baseline`, capture the current state BEFORE deploying.
For each page (either from `--pages` or the homepage): For each page (either from `--pages` or the homepage):
```bash ```bash
$B goto <page-url> aside repl '
$B snapshot -i -a -o ".gstack/canary-reports/baselines/<page-name>.png" const HOOK = `(() => { window.__gstackErrs = window.__gstackErrs || []; const oe = console.error; console.error = (...a) => { window.__gstackErrs.push(a.map(String).join(" ")); oe.apply(console, a); }; window.addEventListener("error", e => window.__gstackErrs.push("uncaught: " + e.message)); window.addEventListener("unhandledrejection", e => window.__gstackErrs.push("unhandledrejection: " + (e.reason && e.reason.message || e.reason))); })()`;
$B console --errors const pg = await openTab("about:blank");
$B perf await pg._sendToTarget("Page.addScriptToEvaluateOnNewDocument", { source: HOOK });
$B text await pg.goto("<page-url>");
console.log("CONSOLE_ERRORS=" + JSON.stringify(await pg.evaluate(() => window.__gstackErrs)));
console.log("NAV=" + await pg.evaluate(() => JSON.stringify(performance.getEntriesByType("navigation")[0])));
console.log("TEXT_START"); console.log((await pg.evaluate(() => document.body.innerText)).slice(0, 20000)); console.log("TEXT_END");
await pg.screenshot({ path: "<page-name>.jpg", type: "jpeg", quality: 60, fullPage: true });
console.log("ASIDE_DIR=" + pwd);
await closeTab(pg);
console.log("GSTACK_STEP_OK");
'
``` ```
Collect for each page: screenshot path, console error count, page load time from `perf`, and a text content snapshot. Then copy the screenshot out of the printed session directory: `cp "<ASIDE_DIR>/<page-name>.jpg" .gstack/canary-reports/baselines/<page-name>.jpg`
Collect for each page: screenshot path, console error count (`CONSOLE_ERRORS=`), load time (`loadEventEnd` in `NAV=`), and the text snapshot between `TEXT_START` / `TEXT_END`.
Save the baseline manifest to `.gstack/canary-reports/baseline.json`: Save the baseline manifest to `.gstack/canary-reports/baseline.json`:
@@ -80,7 +92,7 @@ Save the baseline manifest to `.gstack/canary-reports/baseline.json`:
"branch": "<current branch>", "branch": "<current branch>",
"pages": { "pages": {
"/": { "/": {
"screenshot": "baselines/home.png", "screenshot": "baselines/home.jpg",
"console_errors": 0, "console_errors": 0,
"load_time_ms": 450 "load_time_ms": 450
} }
@@ -95,12 +107,15 @@ Then STOP and tell the user: "Baseline captured. Deploy your changes, then run `
If no `--pages` were specified, auto-discover pages to monitor: If no `--pages` were specified, auto-discover pages to monitor:
```bash ```bash
$B goto <url> aside repl '
$B links const pg = await openTab("<url>");
$B snapshot -i const links = await pg.evaluate(() => [...new Set([...document.querySelectorAll("a[href]")].map(a => a.href))].filter(h => h.startsWith(location.origin) && !/logout|signout|delete|remove|cancel|unsubscribe/i.test(h)));
for (const l of links) { const r = await fetch(l, { method: "HEAD" }).catch(e => ({ status: "ERR " + e.message })); console.log("LINK", r.status, l); }
await closeTab(pg); console.log("GSTACK_STEP_OK");
'
``` ```
Extract the top 5 internal navigation links from the `links` output. Always include the homepage. Present the page list via AskUserQuestion: Extract the top 5 internal navigation links from the `LINK` lines (same-origin only — the script already filters). Always include the homepage. Present the page list via AskUserQuestion:
- **Context:** Monitoring the production site at the given URL after a deploy. - **Context:** Monitoring the production site at the given URL after a deploy.
- **Question:** Which pages should the canary monitor? - **Question:** Which pages should the canary monitor?
@@ -115,29 +130,35 @@ If no `baseline.json` exists, take a quick snapshot now as a reference point.
For each page to monitor: For each page to monitor:
```bash Run the Phase 2 read script for each page with the screenshot saved as `pre-<page-name>.jpg`, then `cp "<ASIDE_DIR>/pre-<page-name>.jpg" .gstack/canary-reports/screenshots/`.
$B goto <page-url>
$B snapshot -i -a -o ".gstack/canary-reports/screenshots/pre-<page-name>.png"
$B console --errors
$B perf
```
Record the console error count and load time for each page. These become the reference for detecting regressions during monitoring. Record the console error count and load time for each page. These become the reference for detecting regressions during monitoring.
### Phase 5: Continuous Monitoring Loop ### Phase 5: Continuous Monitoring Loop
Monitor for the specified duration. Every 60 seconds, check each page: Monitor for the specified duration. Every 60 seconds, check each page. Nothing persists between scripts — every check re-opens the page from its URL and captures fresh evidence:
```bash ```bash
$B goto <page-url> aside repl '
$B snapshot -i -a -o ".gstack/canary-reports/screenshots/<page-name>-<check-number>.png" const HOOK = `(() => { window.__gstackErrs = window.__gstackErrs || []; const oe = console.error; console.error = (...a) => { window.__gstackErrs.push(a.map(String).join(" ")); oe.apply(console, a); }; window.addEventListener("error", e => window.__gstackErrs.push("uncaught: " + e.message)); window.addEventListener("unhandledrejection", e => window.__gstackErrs.push("unhandledrejection: " + (e.reason && e.reason.message || e.reason))); })()`;
$B console --errors const pg = await openTab("about:blank");
$B perf await pg._sendToTarget("Page.addScriptToEvaluateOnNewDocument", { source: HOOK });
await pg.goto("<page-url>");
console.log("CONSOLE_ERRORS=" + JSON.stringify(await pg.evaluate(() => window.__gstackErrs)));
console.log("NAV=" + await pg.evaluate(() => JSON.stringify(performance.getEntriesByType("navigation")[0])));
console.log("TEXT_START"); console.log((await pg.evaluate(() => document.body.innerText)).slice(0, 20000)); console.log("TEXT_END");
await pg.screenshot({ path: "<page-name>-<check-number>.jpg", type: "jpeg", quality: 60, fullPage: true });
console.log("ASIDE_DIR=" + pwd);
await closeTab(pg);
console.log("GSTACK_STEP_OK");
'
``` ```
Then `cp "<ASIDE_DIR>/<page-name>-<check-number>.jpg" .gstack/canary-reports/screenshots/`.
After each check, compare results against the baseline (or pre-deploy snapshot): After each check, compare results against the baseline (or pre-deploy snapshot):
1. **Page load failure** — `goto` returns error or timeout → CRITICAL ALERT 1. **Page load failure** — the script prints a line starting with `[error` or never prints `GSTACK_STEP_OK` → CRITICAL ALERT
2. **New console errors** — errors not present in baseline → HIGH ALERT 2. **New console errors** — errors not present in baseline → HIGH ALERT
3. **Performance regression** — load time exceeds 2x baseline → MEDIUM ALERT 3. **Performance regression** — load time exceeds 2x baseline → MEDIUM ALERT
4. **Broken links** — new 404s not in baseline → LOW ALERT 4. **Broken links** — new 404s not in baseline → LOW ALERT
+28 -16
View File
@@ -3,8 +3,8 @@ name: devex-review
preamble-tier: 3 preamble-tier: 3
version: 1.0.0 version: 1.0.0
description: | description: |
Live developer experience audit. Uses the browse tool to actually TEST the Live developer experience audit. Actually TESTS the developer experience
developer experience: navigates docs, tries the getting started flow, times in the Aside browser: navigates docs, tries the getting started flow, times
TTHW, screenshots error messages, evaluates CLI help text. Produces a DX TTHW, screenshots error messages, evaluates CLI help text. Produces a DX
scorecard with evidence. Compares against /plan-devex-review scores if they scorecard with evidence. Compares against /plan-devex-review scores if they
exist (the boomerang: plan said 3 minutes, reality says 8). Use when asked to exist (the boomerang: plan said 3 minutes, reality says 8). Use when asked to
@@ -33,26 +33,31 @@ allowed-tools:
{{BASE_BRANCH_DETECT}} {{BASE_BRANCH_DETECT}}
{{BROWSE_SETUP}} {{ASIDE_SETUP}}
{{BROWSE_FALLBACK}}
{{ASIDE_COOKBOOK}}
# /devex-review: Live Developer Experience Audit # /devex-review: Live Developer Experience Audit
You are a DX engineer dogfooding a live developer product. Not reviewing a plan. You are a DX engineer dogfooding a live developer product. Not reviewing a plan.
Not reading about the experience. TESTING it. Not reading about the experience. TESTING it.
Use the browse tool to navigate docs, try the getting started flow, and screenshot Drive the Aside browser to navigate docs, try the getting started flow, and screenshot
what developers actually see. Use bash to try CLI commands. Measure, don't guess. what developers actually see. One `aside repl` script per flow, each re-opening from the URL. Use bash to try CLI commands. Measure, don't guess.
{{DX_FRAMEWORK}} {{DX_FRAMEWORK}}
## Scope Declaration ## Scope Declaration
Browse can test web-accessible surfaces: docs pages, API playgrounds, web dashboards, Aside can test web-accessible surfaces: docs pages, API playgrounds, web dashboards,
signup flows, interactive tutorials, error pages. signup flows, interactive tutorials, error pages — with the user's real logged-in
sessions.
Browse CANNOT test: CLI install friction, terminal output quality, local environment Aside CANNOT test: CLI install friction, terminal output quality, local environment
setup, email verification flows, auth requiring real credentials, offline behavior, setup, email verification flows, credential entry (the user signs in themselves; you
build times, IDE integration. never type passwords), offline behavior, build times, IDE integration.
For untestable dimensions, use bash (for CLI --help, README, CHANGELOG) or mark as For untestable dimensions, use bash (for CLI --help, README, CHANGELOG) or mark as
INFERRED from artifacts. Never guess. State your evidence source for every score. INFERRED from artifacts. Never guess. State your evidence source for every score.
@@ -78,7 +83,8 @@ If prior scores exist, display them. These are your baseline for the boomerang c
## Step 1: Getting Started Audit ## Step 1: Getting Started Audit
Navigate to the docs/landing page via browse. Screenshot it. Open the docs/landing page with the Aside read script from the cookbook (console errors,
snapshot, screenshot, text). Copy the screenshot out of the printed ASIDE_DIR and Read it.
``` ```
GETTING STARTED AUDIT GETTING STARTED AUDIT
@@ -95,7 +101,7 @@ Score 0-10. Load "## Pass 1" from dx-hall-of-fame.md for calibration.
Test what you can: Test what you can:
- CLI: Run `--help` via bash. Evaluate output quality, flag design, discoverability. - CLI: Run `--help` via bash. Evaluate output quality, flag design, discoverability.
- API playground: Navigate via browse if one exists. Screenshot. - API playground: Open it in Aside if one exists. Screenshot.
- Naming: Check consistency across the API surface. - Naming: Check consistency across the API surface.
Score 0-10. Load "## Pass 2" from dx-hall-of-fame.md for calibration. Score 0-10. Load "## Pass 2" from dx-hall-of-fame.md for calibration.
@@ -103,7 +109,8 @@ Score 0-10. Load "## Pass 2" from dx-hall-of-fame.md for calibration.
## Step 3: Error Message Audit ## Step 3: Error Message Audit
Trigger common error scenarios: Trigger common error scenarios:
- Browse: Navigate to 404 pages, submit invalid forms, try unauthenticated access - Aside: Open a 404 URL, submit an invalid form (on a non-LOCAL target that is a mutating
action — one AskUserQuestion per run first, per the browser rules), open a protected URL
- CLI: Run with missing args, invalid flags, bad input - CLI: Run with missing args, invalid flags, bad input
Screenshot each error. Score against the Elm/Rust/Stripe three-tier model. Screenshot each error. Score against the Elm/Rust/Stripe three-tier model.
@@ -112,7 +119,9 @@ Score 0-10. Load "## Pass 3" from dx-hall-of-fame.md for calibration.
## Step 4: Documentation Audit ## Step 4: Documentation Audit
Navigate the docs structure via browse: Navigate the docs structure in Aside (search is `pg.fill(<search selector>, <query>)`,
then `pg.locator(<search selector>).press("Enter")` — or `pg.getByRole("searchbox").press("Enter")`,
or a click — then `snapshot`):
- Check search functionality (try 3 common queries) - Check search functionality (try 3 common queries)
- Verify code examples are copy-paste-complete - Verify code examples are copy-paste-complete
- Check language switcher behavior - Check language switcher behavior
@@ -141,12 +150,15 @@ Score 0-10. Evidence: INFERRED from files. Load "## Pass 6" from dx-hall-of-fame
## Step 7: Community & Ecosystem Audit ## Step 7: Community & Ecosystem Audit
Browse: Check the community links the docs point to. Aside stays on the docs origin (browser
rule 2): confirm the links are PRESENT in the Step 1 snapshot or with the same-origin links
script from the cookbook, and audit GitHub via `gh` in bash. Do not open Discord, Stack
Overflow, or any other third-party site — mark those INFERRED (link present, not followed):
- Community links (GitHub Discussions, Discord, Stack Overflow) - Community links (GitHub Discussions, Discord, Stack Overflow)
- GitHub issues (response time, templates, labels) - GitHub issues (response time, templates, labels)
- Contributing guide - Contributing guide
Score 0-10. Evidence: TESTED where web-accessible, INFERRED otherwise. Score 0-10. Evidence: TESTED for the docs page and GitHub, INFERRED otherwise.
## Step 8: DX Measurement Audit ## Step 8: DX Measurement Audit
+30 -25
View File
@@ -24,7 +24,9 @@ triggers:
{{THIRD_PARTY_ACTIONS}} {{THIRD_PARTY_ACTIONS}}
{{BROWSE_SETUP}} {{ASIDE_SETUP}}
{{BROWSE_FALLBACK}}
{{BASE_BRANCH_DETECT}} {{BASE_BRANCH_DETECT}}
@@ -308,45 +310,48 @@ Use the diff-scope classification from Step 5 to determine canary depth:
| Diff Scope | Canary Depth | | Diff Scope | Canary Depth |
|------------|-------------| |------------|-------------|
| SCOPE_DOCS only | Already skipped in Step 5 | | SCOPE_DOCS only | Already skipped in Step 5 |
| SCOPE_CONFIG only | Smoke: `$B goto` + verify 200 status | | SCOPE_CONFIG only | Smoke: the Aside script below; `responseStatus` in `NAV=` must be 200 |
| SCOPE_BACKEND only | Console errors + perf check | | SCOPE_BACKEND only | Console errors + perf check |
| SCOPE_FRONTEND (any) | Full: console + perf + screenshot | | SCOPE_FRONTEND (any) | Full: console + perf + screenshot |
| Mixed scopes | Full canary | | Mixed scopes | Full canary |
**Full canary sequence:** **Full canary sequence** — one `aside repl` script does the whole check (console hook first, then load, then evidence):
```bash ```bash
$B goto <url> aside repl '
const HOOK = `(() => { window.__gstackErrs = window.__gstackErrs || []; const oe = console.error; console.error = (...a) => { window.__gstackErrs.push(a.map(String).join(" ")); oe.apply(console, a); }; window.addEventListener("error", e => window.__gstackErrs.push("uncaught: " + e.message)); window.addEventListener("unhandledrejection", e => window.__gstackErrs.push("unhandledrejection: " + (e.reason && e.reason.message || e.reason))); })()`;
const pg = await openTab("about:blank");
await pg._sendToTarget("Page.addScriptToEvaluateOnNewDocument", { source: HOOK });
await pg.goto("<url>");
console.log("URL=" + pg.url());
console.log("CONSOLE_ERRORS=" + JSON.stringify(await pg.evaluate(() => window.__gstackErrs)));
console.log("NAV=" + await pg.evaluate(() => JSON.stringify(performance.getEntriesByType("navigation")[0])));
console.log("TEXT_START"); console.log((await pg.evaluate(() => document.body.innerText)).slice(0, 20000)); console.log("TEXT_END");
await pg.screenshot({ path: "post-deploy.jpg", type: "jpeg", quality: 60, fullPage: true });
const a = await annotatedScreenshot(pg);
await fs.writeFile(path.join(pwd, "post-deploy-annotated.png"), Buffer.from(a.base64Image, "base64"));
console.log("ASIDE_DIR=" + pwd);
await closeTab(pg);
console.log("GSTACK_STEP_OK");
'
``` ```
Check that the page loaded successfully (200, not an error page). Then copy the evidence out of the printed session directory:
```bash ```bash
$B console --errors mkdir -p .gstack/deploy-reports && cp "<ASIDE_DIR>/post-deploy.jpg" "<ASIDE_DIR>/post-deploy-annotated.png" .gstack/deploy-reports/
``` ```
Check for critical console errors: lines containing `Error`, `Uncaught`, `Failed to load`, `TypeError`, `ReferenceError`. Ignore warnings. Read the output line by line:
```bash - `URL=` — the page loaded and stayed on the site (not a redirect to an error page). A line starting with `[error` or a missing `GSTACK_STEP_OK` means the load failed.
$B perf - `CONSOLE_ERRORS=` — check for critical errors: entries containing `Error`, `Uncaught`, `Failed to load`, `TypeError`, `ReferenceError`. Ignore warnings.
``` - `NAV=` — `responseStatus` is the HTTP status of the document (Chromium PerformanceNavigationTiming) — must be 200. `loadEventEnd` is the page load time. Check that it is under 10 seconds.
- `TEXT_START` / `TEXT_END` — verify the page has real content (not blank, not a generic error page).
Check that page load time is under 10 seconds. - `post-deploy.jpg` and the annotated `post-deploy-annotated.png` are the evidence. Read the copied screenshot so the user sees it.
```bash
$B text
```
Verify the page has content (not blank, not a generic error page).
```bash
$B snapshot -i -a -o ".gstack/deploy-reports/post-deploy.png"
```
Take an annotated screenshot as evidence.
**Health assessment:** **Health assessment:**
- Page loads successfully with 200 status → PASS - Page loads successfully with 200 status (`responseStatus` in `NAV=`) → PASS
- No critical console errors → PASS - No critical console errors → PASS
- Page has real content (not blank or error screen) → PASS - Page has real content (not blank or error screen) → PASS
- Loads in under 10 seconds → PASS - Loads in under 10 seconds → PASS
+4 -4
View File
@@ -279,7 +279,7 @@ describeIfSelected('Canary skill E2E', ['canary-workflow'], () => {
const result = await runSkillTest({ const result = await runSkillTest({
prompt: `Read canary/SKILL.md for the /canary skill instructions. prompt: `Read canary/SKILL.md for the /canary skill instructions.
You are simulating a canary check. There is NO browse daemon available and NO production URL. You are simulating a canary check. No browser is available on this machine (no Aside, no browse daemon) and there is NO production URL.
Instead, demonstrate you understand the workflow: Instead, demonstrate you understand the workflow:
1. Create the .gstack/canary-reports/ directory structure 1. Create the .gstack/canary-reports/ directory structure
@@ -290,7 +290,7 @@ Instead, demonstrate you understand the workflow:
the Phase 6 Health Report format (CANARY REPORT header, duration, pages, status, the Phase 6 Health Report format (CANARY REPORT header, duration, pages, status,
per-page results table, verdict) per-page results table, verdict)
Do NOT use AskUserQuestion. Do NOT run browse ($B) commands. Do NOT use AskUserQuestion. Do NOT run aside or browse ($B) commands.
Just create the directory structure and report files showing the correct schema.`, Just create the directory structure and report files showing the correct schema.`,
workingDirectory: canaryDir, workingDirectory: canaryDir,
maxTurns: 15, maxTurns: 15,
@@ -340,7 +340,7 @@ describeIfSelected('Benchmark skill E2E', ['benchmark-workflow'], () => {
const result = await runSkillTest({ const result = await runSkillTest({
prompt: `Read benchmark/SKILL.md for the /benchmark skill instructions. prompt: `Read benchmark/SKILL.md for the /benchmark skill instructions.
You are simulating a benchmark run. There is NO browse daemon available and NO production URL. You are simulating a benchmark run. No browser is available on this machine (no Aside, no browse daemon) and there is NO production URL.
Instead, demonstrate you understand the workflow: Instead, demonstrate you understand the workflow:
1. Create the .gstack/benchmark-reports/ directory structure including baselines/ 1. Create the .gstack/benchmark-reports/ directory structure including baselines/
@@ -353,7 +353,7 @@ Instead, demonstrate you understand the workflow:
table with Baseline/Current/Delta/Status columns, regression thresholds applied) table with Baseline/Current/Delta/Status columns, regression thresholds applied)
4. Include the Phase 7 Performance Budget section in the report 4. Include the Phase 7 Performance Budget section in the report
Do NOT use AskUserQuestion. Do NOT run browse ($B) commands. Do NOT use AskUserQuestion. Do NOT run aside or browse ($B) commands.
Just create the files showing the correct schema and report format.`, Just create the files showing the correct schema and report format.`,
workingDirectory: benchDir, workingDirectory: benchDir,
maxTurns: 15, maxTurns: 15,