mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-17 18:32:19 +02:00
refactor(deploy): benchmark, canary, land-and-deploy Step 7, devex-review drive Aside
One aside repl script per page prints NAV/PAINT/LCP/RESOURCES/SCRIPTS/CSS/SUMMARY (benchmark), CONSOLE_ERRORS/NAV/TEXT + screenshot (canary, re-run every 60s), and the post-deploy check reads responseStatus from the navigation entry; each carries the $B fallback. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
+28
-16
@@ -3,8 +3,8 @@ name: devex-review
|
||||
preamble-tier: 3
|
||||
version: 1.0.0
|
||||
description: |
|
||||
Live developer experience audit. Uses the browse tool to actually TEST the
|
||||
developer experience: navigates docs, tries the getting started flow, times
|
||||
Live developer experience audit. Actually TESTS the developer experience
|
||||
in the Aside browser: navigates docs, tries the getting started flow, times
|
||||
TTHW, screenshots error messages, evaluates CLI help text. Produces a DX
|
||||
scorecard with evidence. Compares against /plan-devex-review scores if they
|
||||
exist (the boomerang: plan said 3 minutes, reality says 8). Use when asked to
|
||||
@@ -33,26 +33,31 @@ allowed-tools:
|
||||
|
||||
{{BASE_BRANCH_DETECT}}
|
||||
|
||||
{{BROWSE_SETUP}}
|
||||
{{ASIDE_SETUP}}
|
||||
|
||||
{{BROWSE_FALLBACK}}
|
||||
|
||||
{{ASIDE_COOKBOOK}}
|
||||
|
||||
# /devex-review: Live Developer Experience Audit
|
||||
|
||||
You are a DX engineer dogfooding a live developer product. Not reviewing a plan.
|
||||
Not reading about the experience. TESTING it.
|
||||
|
||||
Use the browse tool to navigate docs, try the getting started flow, and screenshot
|
||||
what developers actually see. Use bash to try CLI commands. Measure, don't guess.
|
||||
Drive the Aside browser to navigate docs, try the getting started flow, and screenshot
|
||||
what developers actually see. One `aside repl` script per flow, each re-opening from the URL. Use bash to try CLI commands. Measure, don't guess.
|
||||
|
||||
{{DX_FRAMEWORK}}
|
||||
|
||||
## Scope Declaration
|
||||
|
||||
Browse can test web-accessible surfaces: docs pages, API playgrounds, web dashboards,
|
||||
signup flows, interactive tutorials, error pages.
|
||||
Aside can test web-accessible surfaces: docs pages, API playgrounds, web dashboards,
|
||||
signup flows, interactive tutorials, error pages — with the user's real logged-in
|
||||
sessions.
|
||||
|
||||
Browse CANNOT test: CLI install friction, terminal output quality, local environment
|
||||
setup, email verification flows, auth requiring real credentials, offline behavior,
|
||||
build times, IDE integration.
|
||||
Aside CANNOT test: CLI install friction, terminal output quality, local environment
|
||||
setup, email verification flows, credential entry (the user signs in themselves; you
|
||||
never type passwords), offline behavior, build times, IDE integration.
|
||||
|
||||
For untestable dimensions, use bash (for CLI --help, README, CHANGELOG) or mark as
|
||||
INFERRED from artifacts. Never guess. State your evidence source for every score.
|
||||
@@ -78,7 +83,8 @@ If prior scores exist, display them. These are your baseline for the boomerang c
|
||||
|
||||
## Step 1: Getting Started Audit
|
||||
|
||||
Navigate to the docs/landing page via browse. Screenshot it.
|
||||
Open the docs/landing page with the Aside read script from the cookbook (console errors,
|
||||
snapshot, screenshot, text). Copy the screenshot out of the printed ASIDE_DIR and Read it.
|
||||
|
||||
```
|
||||
GETTING STARTED AUDIT
|
||||
@@ -95,7 +101,7 @@ Score 0-10. Load "## Pass 1" from dx-hall-of-fame.md for calibration.
|
||||
|
||||
Test what you can:
|
||||
- CLI: Run `--help` via bash. Evaluate output quality, flag design, discoverability.
|
||||
- API playground: Navigate via browse if one exists. Screenshot.
|
||||
- API playground: Open it in Aside if one exists. Screenshot.
|
||||
- Naming: Check consistency across the API surface.
|
||||
|
||||
Score 0-10. Load "## Pass 2" from dx-hall-of-fame.md for calibration.
|
||||
@@ -103,7 +109,8 @@ Score 0-10. Load "## Pass 2" from dx-hall-of-fame.md for calibration.
|
||||
## Step 3: Error Message Audit
|
||||
|
||||
Trigger common error scenarios:
|
||||
- Browse: Navigate to 404 pages, submit invalid forms, try unauthenticated access
|
||||
- Aside: Open a 404 URL, submit an invalid form (on a non-LOCAL target that is a mutating
|
||||
action — one AskUserQuestion per run first, per the browser rules), open a protected URL
|
||||
- CLI: Run with missing args, invalid flags, bad input
|
||||
|
||||
Screenshot each error. Score against the Elm/Rust/Stripe three-tier model.
|
||||
@@ -112,7 +119,9 @@ Score 0-10. Load "## Pass 3" from dx-hall-of-fame.md for calibration.
|
||||
|
||||
## Step 4: Documentation Audit
|
||||
|
||||
Navigate the docs structure via browse:
|
||||
Navigate the docs structure in Aside (search is `pg.fill(<search selector>, <query>)`,
|
||||
then `pg.locator(<search selector>).press("Enter")` — or `pg.getByRole("searchbox").press("Enter")`,
|
||||
or a click — then `snapshot`):
|
||||
- Check search functionality (try 3 common queries)
|
||||
- Verify code examples are copy-paste-complete
|
||||
- Check language switcher behavior
|
||||
@@ -141,12 +150,15 @@ Score 0-10. Evidence: INFERRED from files. Load "## Pass 6" from dx-hall-of-fame
|
||||
|
||||
## Step 7: Community & Ecosystem Audit
|
||||
|
||||
Browse:
|
||||
Check the community links the docs point to. Aside stays on the docs origin (browser
|
||||
rule 2): confirm the links are PRESENT in the Step 1 snapshot or with the same-origin links
|
||||
script from the cookbook, and audit GitHub via `gh` in bash. Do not open Discord, Stack
|
||||
Overflow, or any other third-party site — mark those INFERRED (link present, not followed):
|
||||
- Community links (GitHub Discussions, Discord, Stack Overflow)
|
||||
- GitHub issues (response time, templates, labels)
|
||||
- Contributing guide
|
||||
|
||||
Score 0-10. Evidence: TESTED where web-accessible, INFERRED otherwise.
|
||||
Score 0-10. Evidence: TESTED for the docs page and GitHub, INFERRED otherwise.
|
||||
|
||||
## Step 8: DX Measurement Audit
|
||||
|
||||
|
||||
Reference in New Issue
Block a user