mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-04 15:57:33 +01:00
feat(dx): agent-ready error contract for the DevEx review skills
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
1 parent
a889ef9393
commit
2a17a595bf
20 files changed
+467
-81
No files matched your search
@@ -0,0 +1,65 @@
|
||||
# Agent-ready error messages
|
||||
|
||||
gstack's DX reviews (`/plan-devex-review` Pass 3, the DX phase of `/autoplan`, and
|
||||
`/devex-review` Step 3) score every error message for the reader that usually meets it
|
||||
first: an AI agent running unattended. It reads the message literally, cannot ask what was
|
||||
meant, and will retry or give up based on what the message says. This page is the contract
|
||||
those reviews cite (rubric `agent-ready-v1`). The text below is generated from
|
||||
`lib/dx-error-contract.ts`; edit it there.
|
||||
|
||||
| Item | The error gives the reader |
|
||||
|------|----------------------------|
|
||||
| **what** | The operation that failed and the actual values (input, path, id) that caused it. |
|
||||
| **fix** | On the first or last line, never mid-chain: an exact command, flag or edit; or, when no safe agent repair exists, the precise thing a human must provide (credential, approval, decision). A precise escalation is a complete fix. A fix counts only when it is safe and authorized: it does not bypass a consent, safety or sandbox boundary, does not tell the agent to run text copied from untrusted output, and quotes arguments correctly. |
|
||||
| **recovery** | Retry unchanged, fix and retry, do X first, or ask a human; side effects (none, partial, complete, unknown); for an unknown outcome, how to reconcile (status command, idempotency key) before retrying; a retry bound, then escalate. |
|
||||
| **code** | A stable code plus a doc anchor or `--explain` target. Prose may change; codes may not. |
|
||||
| **context** | Versions, the config in effect, relevant env var names (never values), the underlying dependency error, not swallowed. Remote services give caller-safe facts only (parameter, API version, request id). |
|
||||
| **trace** | Unexpected failures only. A local CLI, SDK or skill prints a compact trace or the path of a file holding the full one (full under `--verbose`/`DEBUG`); a remote service returns a correlation id and keeps internals in its logs. |
|
||||
| **redaction** | No secret values, tokens or credential-bearing URLs anywhere, traces and nested causes included. |
|
||||
| **channel** | The surface's native machine signal carries the same fields, and a failure is signalled as a failure (never exit 0, HTTP 200 or `isError: false`). |
|
||||
|
||||
**Failure class.** A usage or validation error rejected before any work began needs what, fix,
|
||||
code, redaction and channel; its recovery is implied (fix and retry, no side effects), and
|
||||
context, reconcile and retry bounds do not apply. Environment, dependency, partial-work and
|
||||
unexpected failures need every applicable item. Trace applies to unexpected failures only.
|
||||
|
||||
**Surface.** Keyed to the product types from the Applicability Gate:
|
||||
- **CLI Tool:** channel is stderr plus a distinct non-zero exit, and a `--json` form when the tool offers structured output.
|
||||
- **API/Service:** channel is HTTP status plus an RFC 9457 problem-details body; context is caller-safe; trace is a correlation id.
|
||||
- **Library/SDK:** channel is a typed error class carrying the code and fields; no exit code.
|
||||
- **Platform:** the CLI or API rule for whichever interface the error reaches.
|
||||
- **Documentation:** what, fix, code and redaction only.
|
||||
- **Claude Code Skill:** channel is the skill's printed result line and code. An MCP tool (a sub-case) returns `isError: true` with the fields in text or structured content.
|
||||
|
||||
**Evidence.** Each item on each path is pass, fail, N/A (with a reason) or unknown. In a plan
|
||||
review, an applicable item the plan never specifies for a path it introduces is a gap (fail);
|
||||
unknown is reserved for existing behavior you could not inspect. In a live audit, error classes
|
||||
you never triggered are unknown, not extrapolated.
|
||||
|
||||
**Caps.** Name the path and item behind every cap and show the corrected error line.
|
||||
- **4/10 (Critical finding, title `CAP4:<path>:<item>`):** a path leaks a secret, signals failure as success, gives an unsafe fix, or blocks waiting for a terminal (hangs, or fails for lack of a terminal without naming a non-interactive alternative such as `--yes`).
|
||||
- **6/10 (High finding, title `CAP6:<path>:<item>`):** a path lacks an applicable fix, recovery, context or trace item.
|
||||
|
||||
Every mode, DX TRIAGE included, reports every CAP4 finding.
|
||||
|
||||
## Examples
|
||||
|
||||
Calibration examples (Elm, Rust, Stripe, a complete CLI error with its RFC 9457 `--json`
|
||||
form, and gstack's own partial example) are in the "## Pass 3" section of
|
||||
[plan-devex-review/dx-hall-of-fame.md](../plan-devex-review/dx-hall-of-fame.md).
|
||||
|
||||
## Sources
|
||||
|
||||
- Anthropic, "Writing effective tools for AI agents": prompt-engineer error responses into specific, actionable corrections.
|
||||
- clig.dev, "Errors": put the actionable line where the reader looks; give debug and traceback information for unexpected errors.
|
||||
- CLI Agent Spec (cli-agent-spec.github.io): per-exit-code `retryable` and `side_effects`.
|
||||
- RFC 9457, Problem Details for HTTP APIs: `type`, `title`, `detail`, `instance`, and extension members.
|
||||
- MCP specification, Tools: tool-execution errors return as `isError: true` results the model can read.
|
||||
- Nx, "Agentic Experience Is the New Developer Experience"; agentsurface.dev, "Designing Errors for Agent Recovery".
|
||||
|
||||
## Live audits: gstack-error-capture
|
||||
|
||||
`/devex-review` triggers CLI errors through `bin/gstack-error-capture`, which runs the
|
||||
target with no terminal, stdin closed, a temporary cwd and HOME, a scrubbed environment and a
|
||||
deadline, and prints one JSON record with redacted streams. `gstack-error-capture --help`
|
||||
lists its flags, record fields and error codes.
|
||||
+2
-2
@@ -30,8 +30,8 @@ Detailed guides for every gstack skill — philosophy, workflow, and examples.
|
||||
| [`/browse`](#browse) | **QA Engineer** | Give the agent eyes. Drives your Aside browser first — real sessions, real clicks, real screenshots — through deterministic `aside repl` scripts, and falls back to gstack's own Chromium (~100ms per command) when Aside isn't there. |
|
||||
| [`/setup-browser-cookies`](#setup-browser-cookies) | **Session Manager** | Copy selected cookies from Chrome, Chromium, Brave, Edge, Windows-only Opera and Opera GX, or macOS-only Comet, Arc, and Dia into the fallback browser. Choose your profile and domains; check sign-in separately. Unnecessary on Aside, which already has your sessions. |
|
||||
| [`/autoplan`](#autoplan) | **Review Pipeline** | One command, fully reviewed plan. Runs CEO → design → DX → eng review automatically (eng always last, so the shipping gate reviews the final amended plan) with encoded decision principles. Surfaces only taste decisions for your approval. |
|
||||
| [`/plan-devex-review`](#plan-devex-review) | **DX Reviewer** | Plan-stage DX review. TTHW (time-to-hello-world), magical moments, friction points, persona traces. Three modes: Expansion, Polish, Triage. |
|
||||
| [`/devex-review`](#devex-review) | **DX Reviewer (live)** | Live developer experience audit. Walks the actual onboarding flow, measures TTHW, catches the docs lies. |
|
||||
| [`/plan-devex-review`](#plan-devex-review) | **DX Reviewer** | Plan-stage DX review. TTHW (time-to-hello-world), magical moments, friction points, persona traces, and error messages scored for the agents that read them ([agent-ready errors](agent-ready-errors.md)). Three modes: Expansion, Polish, Triage. |
|
||||
| [`/devex-review`](#devex-review) | **DX Reviewer (live)** | Live developer experience audit. Walks the actual onboarding flow, measures TTHW, catches the docs lies, and triggers CLI errors the way an agent meets them (no terminal, contained, redacted) through `gstack-error-capture`. |
|
||||
| [`/plan-tune`](#plan-tune) | **Question Tuner** | Self-tune AskUserQuestion sensitivity per question. Mark questions as never-ask, always-ask, or only-for-one-way. |
|
||||
| [`/spec`](#spec) | **Spec Author** | Turn vague intent into a precise, executable spec in five phases. Files a GitHub issue, optionally spawns a Claude Code agent in a fresh worktree, and lets `/ship` close the source issue on merge. |
|
||||
| [`/learn`](#learn) | **Memory** | Manage what gstack learned across sessions. Review, search, prune, and export project-specific patterns and preferences. |
|
||||
|
||||
Reference in new issue
Block a user