feat: telemetry epilogue captures error context + regenerate SKILLs

Epilogue now instructs Claude to classify errors (error_class from a
defined taxonomy), write a one-line error_message, and identify the
failed_step. All 33 SKILL.md files regenerated.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-03-20 08:21:00 -07:00
parent 6ef78ab6c8
commit b349769e2e
34 changed files with 646 additions and 102 deletions
+19 -3
View File
@@ -228,7 +228,19 @@ RECOMMENDATION: [what the user should do next]
After the skill workflow completes (success, error, or abort), log the telemetry event.
Determine the skill name from the `name:` field in this file's YAML frontmatter.
Determine the outcome from the workflow result (success if completed normally, error
if it failed, abort if the user interrupted). Run this bash:
if it failed, abort if the user interrupted).
**For errors:** Also determine:
- `ERROR_CLASS`: a short category — one of: `timeout`, `test_failure`, `build_failure`,
`git_error`, `auth_error`, `network_error`, `browse_error`, `lint_error`,
`merge_conflict`, `permission_error`, `unknown_error`. Pick the most specific match.
- `ERROR_MESSAGE`: a one-line summary of what went wrong (max 200 chars). Include the
command that failed and the key error text. Example: `"bun test: 3 tests failed in
auth.test.ts — expected 200 got 401"`. Never include file paths, secrets, or PII.
- `FAILED_STEP`: which step in the skill workflow failed. Example: `"run_tests"`,
`"create_pr"`, `"merge_base"`, `"build"`, `"qa_browse"`. Use snake_case, max 30 chars.
Run this bash:
```bash
_TEL_END=$(date +%s)
@@ -236,12 +248,16 @@ _TEL_DUR=$(( _TEL_END - _TEL_START ))
rm -f ~/.gstack/analytics/.pending-"$_SESSION_ID" 2>/dev/null || true
~/.codex/skills/gstack/bin/gstack-telemetry-log \
--skill "SKILL_NAME" --duration "$_TEL_DUR" --outcome "OUTCOME" \
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" 2>/dev/null &
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" \
--error-class "ERROR_CLASS" --error-message "ERROR_MESSAGE" \
--failed-step "FAILED_STEP" 2>/dev/null &
```
Replace `SKILL_NAME` with the actual skill name from frontmatter, `OUTCOME` with
success/error/abort, and `USED_BROWSE` with true/false based on whether `$B` was used.
If you cannot determine the outcome, use "unknown". This runs in the background and
For `ERROR_CLASS`, `ERROR_MESSAGE`, and `FAILED_STEP`: use empty string `""` if the
outcome is not error. If the outcome is error but you cannot determine the details,
use `"unknown_error"`, `""`, and `""` respectively. This runs in the background and
never blocks the user.
# browse: QA Testing & Dogfooding
@@ -229,7 +229,19 @@ RECOMMENDATION: [what the user should do next]
After the skill workflow completes (success, error, or abort), log the telemetry event.
Determine the skill name from the `name:` field in this file's YAML frontmatter.
Determine the outcome from the workflow result (success if completed normally, error
if it failed, abort if the user interrupted). Run this bash:
if it failed, abort if the user interrupted).
**For errors:** Also determine:
- `ERROR_CLASS`: a short category — one of: `timeout`, `test_failure`, `build_failure`,
`git_error`, `auth_error`, `network_error`, `browse_error`, `lint_error`,
`merge_conflict`, `permission_error`, `unknown_error`. Pick the most specific match.
- `ERROR_MESSAGE`: a one-line summary of what went wrong (max 200 chars). Include the
command that failed and the key error text. Example: `"bun test: 3 tests failed in
auth.test.ts — expected 200 got 401"`. Never include file paths, secrets, or PII.
- `FAILED_STEP`: which step in the skill workflow failed. Example: `"run_tests"`,
`"create_pr"`, `"merge_base"`, `"build"`, `"qa_browse"`. Use snake_case, max 30 chars.
Run this bash:
```bash
_TEL_END=$(date +%s)
@@ -237,12 +249,16 @@ _TEL_DUR=$(( _TEL_END - _TEL_START ))
rm -f ~/.gstack/analytics/.pending-"$_SESSION_ID" 2>/dev/null || true
~/.codex/skills/gstack/bin/gstack-telemetry-log \
--skill "SKILL_NAME" --duration "$_TEL_DUR" --outcome "OUTCOME" \
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" 2>/dev/null &
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" \
--error-class "ERROR_CLASS" --error-message "ERROR_MESSAGE" \
--failed-step "FAILED_STEP" 2>/dev/null &
```
Replace `SKILL_NAME` with the actual skill name from frontmatter, `OUTCOME` with
success/error/abort, and `USED_BROWSE` with true/false based on whether `$B` was used.
If you cannot determine the outcome, use "unknown". This runs in the background and
For `ERROR_CLASS`, `ERROR_MESSAGE`, and `FAILED_STEP`: use empty string `""` if the
outcome is not error. If the outcome is error but you cannot determine the details,
use `"unknown_error"`, `""`, and `""` respectively. This runs in the background and
never blocks the user.
# /design-consultation: Your Design System, Built Together
+19 -3
View File
@@ -229,7 +229,19 @@ RECOMMENDATION: [what the user should do next]
After the skill workflow completes (success, error, or abort), log the telemetry event.
Determine the skill name from the `name:` field in this file's YAML frontmatter.
Determine the outcome from the workflow result (success if completed normally, error
if it failed, abort if the user interrupted). Run this bash:
if it failed, abort if the user interrupted).
**For errors:** Also determine:
- `ERROR_CLASS`: a short category — one of: `timeout`, `test_failure`, `build_failure`,
`git_error`, `auth_error`, `network_error`, `browse_error`, `lint_error`,
`merge_conflict`, `permission_error`, `unknown_error`. Pick the most specific match.
- `ERROR_MESSAGE`: a one-line summary of what went wrong (max 200 chars). Include the
command that failed and the key error text. Example: `"bun test: 3 tests failed in
auth.test.ts — expected 200 got 401"`. Never include file paths, secrets, or PII.
- `FAILED_STEP`: which step in the skill workflow failed. Example: `"run_tests"`,
`"create_pr"`, `"merge_base"`, `"build"`, `"qa_browse"`. Use snake_case, max 30 chars.
Run this bash:
```bash
_TEL_END=$(date +%s)
@@ -237,12 +249,16 @@ _TEL_DUR=$(( _TEL_END - _TEL_START ))
rm -f ~/.gstack/analytics/.pending-"$_SESSION_ID" 2>/dev/null || true
~/.codex/skills/gstack/bin/gstack-telemetry-log \
--skill "SKILL_NAME" --duration "$_TEL_DUR" --outcome "OUTCOME" \
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" 2>/dev/null &
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" \
--error-class "ERROR_CLASS" --error-message "ERROR_MESSAGE" \
--failed-step "FAILED_STEP" 2>/dev/null &
```
Replace `SKILL_NAME` with the actual skill name from frontmatter, `OUTCOME` with
success/error/abort, and `USED_BROWSE` with true/false based on whether `$B` was used.
If you cannot determine the outcome, use "unknown". This runs in the background and
For `ERROR_CLASS`, `ERROR_MESSAGE`, and `FAILED_STEP`: use empty string `""` if the
outcome is not error. If the outcome is error but you cannot determine the details,
use `"unknown_error"`, `""`, and `""` respectively. This runs in the background and
never blocks the user.
# /design-review: Design Audit → Fix → Verify
@@ -227,7 +227,19 @@ RECOMMENDATION: [what the user should do next]
After the skill workflow completes (success, error, or abort), log the telemetry event.
Determine the skill name from the `name:` field in this file's YAML frontmatter.
Determine the outcome from the workflow result (success if completed normally, error
if it failed, abort if the user interrupted). Run this bash:
if it failed, abort if the user interrupted).
**For errors:** Also determine:
- `ERROR_CLASS`: a short category — one of: `timeout`, `test_failure`, `build_failure`,
`git_error`, `auth_error`, `network_error`, `browse_error`, `lint_error`,
`merge_conflict`, `permission_error`, `unknown_error`. Pick the most specific match.
- `ERROR_MESSAGE`: a one-line summary of what went wrong (max 200 chars). Include the
command that failed and the key error text. Example: `"bun test: 3 tests failed in
auth.test.ts — expected 200 got 401"`. Never include file paths, secrets, or PII.
- `FAILED_STEP`: which step in the skill workflow failed. Example: `"run_tests"`,
`"create_pr"`, `"merge_base"`, `"build"`, `"qa_browse"`. Use snake_case, max 30 chars.
Run this bash:
```bash
_TEL_END=$(date +%s)
@@ -235,12 +247,16 @@ _TEL_DUR=$(( _TEL_END - _TEL_START ))
rm -f ~/.gstack/analytics/.pending-"$_SESSION_ID" 2>/dev/null || true
~/.codex/skills/gstack/bin/gstack-telemetry-log \
--skill "SKILL_NAME" --duration "$_TEL_DUR" --outcome "OUTCOME" \
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" 2>/dev/null &
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" \
--error-class "ERROR_CLASS" --error-message "ERROR_MESSAGE" \
--failed-step "FAILED_STEP" 2>/dev/null &
```
Replace `SKILL_NAME` with the actual skill name from frontmatter, `OUTCOME` with
success/error/abort, and `USED_BROWSE` with true/false based on whether `$B` was used.
If you cannot determine the outcome, use "unknown". This runs in the background and
For `ERROR_CLASS`, `ERROR_MESSAGE`, and `FAILED_STEP`: use empty string `""` if the
outcome is not error. If the outcome is error but you cannot determine the details,
use `"unknown_error"`, `""`, and `""` respectively. This runs in the background and
never blocks the user.
## Step 0: Detect base branch
+19 -3
View File
@@ -230,7 +230,19 @@ RECOMMENDATION: [what the user should do next]
After the skill workflow completes (success, error, or abort), log the telemetry event.
Determine the skill name from the `name:` field in this file's YAML frontmatter.
Determine the outcome from the workflow result (success if completed normally, error
if it failed, abort if the user interrupted). Run this bash:
if it failed, abort if the user interrupted).
**For errors:** Also determine:
- `ERROR_CLASS`: a short category — one of: `timeout`, `test_failure`, `build_failure`,
`git_error`, `auth_error`, `network_error`, `browse_error`, `lint_error`,
`merge_conflict`, `permission_error`, `unknown_error`. Pick the most specific match.
- `ERROR_MESSAGE`: a one-line summary of what went wrong (max 200 chars). Include the
command that failed and the key error text. Example: `"bun test: 3 tests failed in
auth.test.ts — expected 200 got 401"`. Never include file paths, secrets, or PII.
- `FAILED_STEP`: which step in the skill workflow failed. Example: `"run_tests"`,
`"create_pr"`, `"merge_base"`, `"build"`, `"qa_browse"`. Use snake_case, max 30 chars.
Run this bash:
```bash
_TEL_END=$(date +%s)
@@ -238,12 +250,16 @@ _TEL_DUR=$(( _TEL_END - _TEL_START ))
rm -f ~/.gstack/analytics/.pending-"$_SESSION_ID" 2>/dev/null || true
~/.codex/skills/gstack/bin/gstack-telemetry-log \
--skill "SKILL_NAME" --duration "$_TEL_DUR" --outcome "OUTCOME" \
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" 2>/dev/null &
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" \
--error-class "ERROR_CLASS" --error-message "ERROR_MESSAGE" \
--failed-step "FAILED_STEP" 2>/dev/null &
```
Replace `SKILL_NAME` with the actual skill name from frontmatter, `OUTCOME` with
success/error/abort, and `USED_BROWSE` with true/false based on whether `$B` was used.
If you cannot determine the outcome, use "unknown". This runs in the background and
For `ERROR_CLASS`, `ERROR_MESSAGE`, and `FAILED_STEP`: use empty string `""` if the
outcome is not error. If the outcome is error but you cannot determine the details,
use `"unknown_error"`, `""`, and `""` respectively. This runs in the background and
never blocks the user.
# Systematic Debugging
+19 -3
View File
@@ -231,7 +231,19 @@ RECOMMENDATION: [what the user should do next]
After the skill workflow completes (success, error, or abort), log the telemetry event.
Determine the skill name from the `name:` field in this file's YAML frontmatter.
Determine the outcome from the workflow result (success if completed normally, error
if it failed, abort if the user interrupted). Run this bash:
if it failed, abort if the user interrupted).
**For errors:** Also determine:
- `ERROR_CLASS`: a short category — one of: `timeout`, `test_failure`, `build_failure`,
`git_error`, `auth_error`, `network_error`, `browse_error`, `lint_error`,
`merge_conflict`, `permission_error`, `unknown_error`. Pick the most specific match.
- `ERROR_MESSAGE`: a one-line summary of what went wrong (max 200 chars). Include the
command that failed and the key error text. Example: `"bun test: 3 tests failed in
auth.test.ts — expected 200 got 401"`. Never include file paths, secrets, or PII.
- `FAILED_STEP`: which step in the skill workflow failed. Example: `"run_tests"`,
`"create_pr"`, `"merge_base"`, `"build"`, `"qa_browse"`. Use snake_case, max 30 chars.
Run this bash:
```bash
_TEL_END=$(date +%s)
@@ -239,12 +251,16 @@ _TEL_DUR=$(( _TEL_END - _TEL_START ))
rm -f ~/.gstack/analytics/.pending-"$_SESSION_ID" 2>/dev/null || true
~/.codex/skills/gstack/bin/gstack-telemetry-log \
--skill "SKILL_NAME" --duration "$_TEL_DUR" --outcome "OUTCOME" \
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" 2>/dev/null &
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" \
--error-class "ERROR_CLASS" --error-message "ERROR_MESSAGE" \
--failed-step "FAILED_STEP" 2>/dev/null &
```
Replace `SKILL_NAME` with the actual skill name from frontmatter, `OUTCOME` with
success/error/abort, and `USED_BROWSE` with true/false based on whether `$B` was used.
If you cannot determine the outcome, use "unknown". This runs in the background and
For `ERROR_CLASS`, `ERROR_MESSAGE`, and `FAILED_STEP`: use empty string `""` if the
outcome is not error. If the outcome is error but you cannot determine the details,
use `"unknown_error"`, `""`, and `""` respectively. This runs in the background and
never blocks the user.
# YC Office Hours
+19 -3
View File
@@ -230,7 +230,19 @@ RECOMMENDATION: [what the user should do next]
After the skill workflow completes (success, error, or abort), log the telemetry event.
Determine the skill name from the `name:` field in this file's YAML frontmatter.
Determine the outcome from the workflow result (success if completed normally, error
if it failed, abort if the user interrupted). Run this bash:
if it failed, abort if the user interrupted).
**For errors:** Also determine:
- `ERROR_CLASS`: a short category — one of: `timeout`, `test_failure`, `build_failure`,
`git_error`, `auth_error`, `network_error`, `browse_error`, `lint_error`,
`merge_conflict`, `permission_error`, `unknown_error`. Pick the most specific match.
- `ERROR_MESSAGE`: a one-line summary of what went wrong (max 200 chars). Include the
command that failed and the key error text. Example: `"bun test: 3 tests failed in
auth.test.ts — expected 200 got 401"`. Never include file paths, secrets, or PII.
- `FAILED_STEP`: which step in the skill workflow failed. Example: `"run_tests"`,
`"create_pr"`, `"merge_base"`, `"build"`, `"qa_browse"`. Use snake_case, max 30 chars.
Run this bash:
```bash
_TEL_END=$(date +%s)
@@ -238,12 +250,16 @@ _TEL_DUR=$(( _TEL_END - _TEL_START ))
rm -f ~/.gstack/analytics/.pending-"$_SESSION_ID" 2>/dev/null || true
~/.codex/skills/gstack/bin/gstack-telemetry-log \
--skill "SKILL_NAME" --duration "$_TEL_DUR" --outcome "OUTCOME" \
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" 2>/dev/null &
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" \
--error-class "ERROR_CLASS" --error-message "ERROR_MESSAGE" \
--failed-step "FAILED_STEP" 2>/dev/null &
```
Replace `SKILL_NAME` with the actual skill name from frontmatter, `OUTCOME` with
success/error/abort, and `USED_BROWSE` with true/false based on whether `$B` was used.
If you cannot determine the outcome, use "unknown". This runs in the background and
For `ERROR_CLASS`, `ERROR_MESSAGE`, and `FAILED_STEP`: use empty string `""` if the
outcome is not error. If the outcome is error but you cannot determine the details,
use `"unknown_error"`, `""`, and `""` respectively. This runs in the background and
never blocks the user.
## Step 0: Detect base branch
@@ -229,7 +229,19 @@ RECOMMENDATION: [what the user should do next]
After the skill workflow completes (success, error, or abort), log the telemetry event.
Determine the skill name from the `name:` field in this file's YAML frontmatter.
Determine the outcome from the workflow result (success if completed normally, error
if it failed, abort if the user interrupted). Run this bash:
if it failed, abort if the user interrupted).
**For errors:** Also determine:
- `ERROR_CLASS`: a short category — one of: `timeout`, `test_failure`, `build_failure`,
`git_error`, `auth_error`, `network_error`, `browse_error`, `lint_error`,
`merge_conflict`, `permission_error`, `unknown_error`. Pick the most specific match.
- `ERROR_MESSAGE`: a one-line summary of what went wrong (max 200 chars). Include the
command that failed and the key error text. Example: `"bun test: 3 tests failed in
auth.test.ts — expected 200 got 401"`. Never include file paths, secrets, or PII.
- `FAILED_STEP`: which step in the skill workflow failed. Example: `"run_tests"`,
`"create_pr"`, `"merge_base"`, `"build"`, `"qa_browse"`. Use snake_case, max 30 chars.
Run this bash:
```bash
_TEL_END=$(date +%s)
@@ -237,12 +249,16 @@ _TEL_DUR=$(( _TEL_END - _TEL_START ))
rm -f ~/.gstack/analytics/.pending-"$_SESSION_ID" 2>/dev/null || true
~/.codex/skills/gstack/bin/gstack-telemetry-log \
--skill "SKILL_NAME" --duration "$_TEL_DUR" --outcome "OUTCOME" \
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" 2>/dev/null &
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" \
--error-class "ERROR_CLASS" --error-message "ERROR_MESSAGE" \
--failed-step "FAILED_STEP" 2>/dev/null &
```
Replace `SKILL_NAME` with the actual skill name from frontmatter, `OUTCOME` with
success/error/abort, and `USED_BROWSE` with true/false based on whether `$B` was used.
If you cannot determine the outcome, use "unknown". This runs in the background and
For `ERROR_CLASS`, `ERROR_MESSAGE`, and `FAILED_STEP`: use empty string `""` if the
outcome is not error. If the outcome is error but you cannot determine the details,
use `"unknown_error"`, `""`, and `""` respectively. This runs in the background and
never blocks the user.
## Step 0: Detect base branch
+19 -3
View File
@@ -228,7 +228,19 @@ RECOMMENDATION: [what the user should do next]
After the skill workflow completes (success, error, or abort), log the telemetry event.
Determine the skill name from the `name:` field in this file's YAML frontmatter.
Determine the outcome from the workflow result (success if completed normally, error
if it failed, abort if the user interrupted). Run this bash:
if it failed, abort if the user interrupted).
**For errors:** Also determine:
- `ERROR_CLASS`: a short category — one of: `timeout`, `test_failure`, `build_failure`,
`git_error`, `auth_error`, `network_error`, `browse_error`, `lint_error`,
`merge_conflict`, `permission_error`, `unknown_error`. Pick the most specific match.
- `ERROR_MESSAGE`: a one-line summary of what went wrong (max 200 chars). Include the
command that failed and the key error text. Example: `"bun test: 3 tests failed in
auth.test.ts — expected 200 got 401"`. Never include file paths, secrets, or PII.
- `FAILED_STEP`: which step in the skill workflow failed. Example: `"run_tests"`,
`"create_pr"`, `"merge_base"`, `"build"`, `"qa_browse"`. Use snake_case, max 30 chars.
Run this bash:
```bash
_TEL_END=$(date +%s)
@@ -236,12 +248,16 @@ _TEL_DUR=$(( _TEL_END - _TEL_START ))
rm -f ~/.gstack/analytics/.pending-"$_SESSION_ID" 2>/dev/null || true
~/.codex/skills/gstack/bin/gstack-telemetry-log \
--skill "SKILL_NAME" --duration "$_TEL_DUR" --outcome "OUTCOME" \
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" 2>/dev/null &
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" \
--error-class "ERROR_CLASS" --error-message "ERROR_MESSAGE" \
--failed-step "FAILED_STEP" 2>/dev/null &
```
Replace `SKILL_NAME` with the actual skill name from frontmatter, `OUTCOME` with
success/error/abort, and `USED_BROWSE` with true/false based on whether `$B` was used.
If you cannot determine the outcome, use "unknown". This runs in the background and
For `ERROR_CLASS`, `ERROR_MESSAGE`, and `FAILED_STEP`: use empty string `""` if the
outcome is not error. If the outcome is error but you cannot determine the details,
use `"unknown_error"`, `""`, and `""` respectively. This runs in the background and
never blocks the user.
# Plan Review Mode
+19 -3
View File
@@ -227,7 +227,19 @@ RECOMMENDATION: [what the user should do next]
After the skill workflow completes (success, error, or abort), log the telemetry event.
Determine the skill name from the `name:` field in this file's YAML frontmatter.
Determine the outcome from the workflow result (success if completed normally, error
if it failed, abort if the user interrupted). Run this bash:
if it failed, abort if the user interrupted).
**For errors:** Also determine:
- `ERROR_CLASS`: a short category — one of: `timeout`, `test_failure`, `build_failure`,
`git_error`, `auth_error`, `network_error`, `browse_error`, `lint_error`,
`merge_conflict`, `permission_error`, `unknown_error`. Pick the most specific match.
- `ERROR_MESSAGE`: a one-line summary of what went wrong (max 200 chars). Include the
command that failed and the key error text. Example: `"bun test: 3 tests failed in
auth.test.ts — expected 200 got 401"`. Never include file paths, secrets, or PII.
- `FAILED_STEP`: which step in the skill workflow failed. Example: `"run_tests"`,
`"create_pr"`, `"merge_base"`, `"build"`, `"qa_browse"`. Use snake_case, max 30 chars.
Run this bash:
```bash
_TEL_END=$(date +%s)
@@ -235,12 +247,16 @@ _TEL_DUR=$(( _TEL_END - _TEL_START ))
rm -f ~/.gstack/analytics/.pending-"$_SESSION_ID" 2>/dev/null || true
~/.codex/skills/gstack/bin/gstack-telemetry-log \
--skill "SKILL_NAME" --duration "$_TEL_DUR" --outcome "OUTCOME" \
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" 2>/dev/null &
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" \
--error-class "ERROR_CLASS" --error-message "ERROR_MESSAGE" \
--failed-step "FAILED_STEP" 2>/dev/null &
```
Replace `SKILL_NAME` with the actual skill name from frontmatter, `OUTCOME` with
success/error/abort, and `USED_BROWSE` with true/false based on whether `$B` was used.
If you cannot determine the outcome, use "unknown". This runs in the background and
For `ERROR_CLASS`, `ERROR_MESSAGE`, and `FAILED_STEP`: use empty string `""` if the
outcome is not error. If the outcome is error but you cannot determine the details,
use `"unknown_error"`, `""`, and `""` respectively. This runs in the background and
never blocks the user.
# /qa-only: Report-Only QA Testing
+19 -3
View File
@@ -230,7 +230,19 @@ RECOMMENDATION: [what the user should do next]
After the skill workflow completes (success, error, or abort), log the telemetry event.
Determine the skill name from the `name:` field in this file's YAML frontmatter.
Determine the outcome from the workflow result (success if completed normally, error
if it failed, abort if the user interrupted). Run this bash:
if it failed, abort if the user interrupted).
**For errors:** Also determine:
- `ERROR_CLASS`: a short category — one of: `timeout`, `test_failure`, `build_failure`,
`git_error`, `auth_error`, `network_error`, `browse_error`, `lint_error`,
`merge_conflict`, `permission_error`, `unknown_error`. Pick the most specific match.
- `ERROR_MESSAGE`: a one-line summary of what went wrong (max 200 chars). Include the
command that failed and the key error text. Example: `"bun test: 3 tests failed in
auth.test.ts — expected 200 got 401"`. Never include file paths, secrets, or PII.
- `FAILED_STEP`: which step in the skill workflow failed. Example: `"run_tests"`,
`"create_pr"`, `"merge_base"`, `"build"`, `"qa_browse"`. Use snake_case, max 30 chars.
Run this bash:
```bash
_TEL_END=$(date +%s)
@@ -238,12 +250,16 @@ _TEL_DUR=$(( _TEL_END - _TEL_START ))
rm -f ~/.gstack/analytics/.pending-"$_SESSION_ID" 2>/dev/null || true
~/.codex/skills/gstack/bin/gstack-telemetry-log \
--skill "SKILL_NAME" --duration "$_TEL_DUR" --outcome "OUTCOME" \
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" 2>/dev/null &
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" \
--error-class "ERROR_CLASS" --error-message "ERROR_MESSAGE" \
--failed-step "FAILED_STEP" 2>/dev/null &
```
Replace `SKILL_NAME` with the actual skill name from frontmatter, `OUTCOME` with
success/error/abort, and `USED_BROWSE` with true/false based on whether `$B` was used.
If you cannot determine the outcome, use "unknown". This runs in the background and
For `ERROR_CLASS`, `ERROR_MESSAGE`, and `FAILED_STEP`: use empty string `""` if the
outcome is not error. If the outcome is error but you cannot determine the details,
use `"unknown_error"`, `""`, and `""` respectively. This runs in the background and
never blocks the user.
## Step 0: Detect base branch
+19 -3
View File
@@ -227,7 +227,19 @@ RECOMMENDATION: [what the user should do next]
After the skill workflow completes (success, error, or abort), log the telemetry event.
Determine the skill name from the `name:` field in this file's YAML frontmatter.
Determine the outcome from the workflow result (success if completed normally, error
if it failed, abort if the user interrupted). Run this bash:
if it failed, abort if the user interrupted).
**For errors:** Also determine:
- `ERROR_CLASS`: a short category — one of: `timeout`, `test_failure`, `build_failure`,
`git_error`, `auth_error`, `network_error`, `browse_error`, `lint_error`,
`merge_conflict`, `permission_error`, `unknown_error`. Pick the most specific match.
- `ERROR_MESSAGE`: a one-line summary of what went wrong (max 200 chars). Include the
command that failed and the key error text. Example: `"bun test: 3 tests failed in
auth.test.ts — expected 200 got 401"`. Never include file paths, secrets, or PII.
- `FAILED_STEP`: which step in the skill workflow failed. Example: `"run_tests"`,
`"create_pr"`, `"merge_base"`, `"build"`, `"qa_browse"`. Use snake_case, max 30 chars.
Run this bash:
```bash
_TEL_END=$(date +%s)
@@ -235,12 +247,16 @@ _TEL_DUR=$(( _TEL_END - _TEL_START ))
rm -f ~/.gstack/analytics/.pending-"$_SESSION_ID" 2>/dev/null || true
~/.codex/skills/gstack/bin/gstack-telemetry-log \
--skill "SKILL_NAME" --duration "$_TEL_DUR" --outcome "OUTCOME" \
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" 2>/dev/null &
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" \
--error-class "ERROR_CLASS" --error-message "ERROR_MESSAGE" \
--failed-step "FAILED_STEP" 2>/dev/null &
```
Replace `SKILL_NAME` with the actual skill name from frontmatter, `OUTCOME` with
success/error/abort, and `USED_BROWSE` with true/false based on whether `$B` was used.
If you cannot determine the outcome, use "unknown". This runs in the background and
For `ERROR_CLASS`, `ERROR_MESSAGE`, and `FAILED_STEP`: use empty string `""` if the
outcome is not error. If the outcome is error but you cannot determine the details,
use `"unknown_error"`, `""`, and `""` respectively. This runs in the background and
never blocks the user.
## Detect default branch
+19 -3
View File
@@ -226,7 +226,19 @@ RECOMMENDATION: [what the user should do next]
After the skill workflow completes (success, error, or abort), log the telemetry event.
Determine the skill name from the `name:` field in this file's YAML frontmatter.
Determine the outcome from the workflow result (success if completed normally, error
if it failed, abort if the user interrupted). Run this bash:
if it failed, abort if the user interrupted).
**For errors:** Also determine:
- `ERROR_CLASS`: a short category — one of: `timeout`, `test_failure`, `build_failure`,
`git_error`, `auth_error`, `network_error`, `browse_error`, `lint_error`,
`merge_conflict`, `permission_error`, `unknown_error`. Pick the most specific match.
- `ERROR_MESSAGE`: a one-line summary of what went wrong (max 200 chars). Include the
command that failed and the key error text. Example: `"bun test: 3 tests failed in
auth.test.ts — expected 200 got 401"`. Never include file paths, secrets, or PII.
- `FAILED_STEP`: which step in the skill workflow failed. Example: `"run_tests"`,
`"create_pr"`, `"merge_base"`, `"build"`, `"qa_browse"`. Use snake_case, max 30 chars.
Run this bash:
```bash
_TEL_END=$(date +%s)
@@ -234,12 +246,16 @@ _TEL_DUR=$(( _TEL_END - _TEL_START ))
rm -f ~/.gstack/analytics/.pending-"$_SESSION_ID" 2>/dev/null || true
~/.codex/skills/gstack/bin/gstack-telemetry-log \
--skill "SKILL_NAME" --duration "$_TEL_DUR" --outcome "OUTCOME" \
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" 2>/dev/null &
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" \
--error-class "ERROR_CLASS" --error-message "ERROR_MESSAGE" \
--failed-step "FAILED_STEP" 2>/dev/null &
```
Replace `SKILL_NAME` with the actual skill name from frontmatter, `OUTCOME` with
success/error/abort, and `USED_BROWSE` with true/false based on whether `$B` was used.
If you cannot determine the outcome, use "unknown". This runs in the background and
For `ERROR_CLASS`, `ERROR_MESSAGE`, and `FAILED_STEP`: use empty string `""` if the
outcome is not error. If the outcome is error but you cannot determine the details,
use `"unknown_error"`, `""`, and `""` respectively. This runs in the background and
never blocks the user.
## Step 0: Detect base branch
@@ -226,7 +226,19 @@ RECOMMENDATION: [what the user should do next]
After the skill workflow completes (success, error, or abort), log the telemetry event.
Determine the skill name from the `name:` field in this file's YAML frontmatter.
Determine the outcome from the workflow result (success if completed normally, error
if it failed, abort if the user interrupted). Run this bash:
if it failed, abort if the user interrupted).
**For errors:** Also determine:
- `ERROR_CLASS`: a short category — one of: `timeout`, `test_failure`, `build_failure`,
`git_error`, `auth_error`, `network_error`, `browse_error`, `lint_error`,
`merge_conflict`, `permission_error`, `unknown_error`. Pick the most specific match.
- `ERROR_MESSAGE`: a one-line summary of what went wrong (max 200 chars). Include the
command that failed and the key error text. Example: `"bun test: 3 tests failed in
auth.test.ts — expected 200 got 401"`. Never include file paths, secrets, or PII.
- `FAILED_STEP`: which step in the skill workflow failed. Example: `"run_tests"`,
`"create_pr"`, `"merge_base"`, `"build"`, `"qa_browse"`. Use snake_case, max 30 chars.
Run this bash:
```bash
_TEL_END=$(date +%s)
@@ -234,12 +246,16 @@ _TEL_DUR=$(( _TEL_END - _TEL_START ))
rm -f ~/.gstack/analytics/.pending-"$_SESSION_ID" 2>/dev/null || true
~/.codex/skills/gstack/bin/gstack-telemetry-log \
--skill "SKILL_NAME" --duration "$_TEL_DUR" --outcome "OUTCOME" \
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" 2>/dev/null &
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" \
--error-class "ERROR_CLASS" --error-message "ERROR_MESSAGE" \
--failed-step "FAILED_STEP" 2>/dev/null &
```
Replace `SKILL_NAME` with the actual skill name from frontmatter, `OUTCOME` with
success/error/abort, and `USED_BROWSE` with true/false based on whether `$B` was used.
If you cannot determine the outcome, use "unknown". This runs in the background and
For `ERROR_CLASS`, `ERROR_MESSAGE`, and `FAILED_STEP`: use empty string `""` if the
outcome is not error. If the outcome is error but you cannot determine the details,
use `"unknown_error"`, `""`, and `""` respectively. This runs in the background and
never blocks the user.
# Setup Browser Cookies
+19 -3
View File
@@ -224,7 +224,19 @@ RECOMMENDATION: [what the user should do next]
After the skill workflow completes (success, error, or abort), log the telemetry event.
Determine the skill name from the `name:` field in this file's YAML frontmatter.
Determine the outcome from the workflow result (success if completed normally, error
if it failed, abort if the user interrupted). Run this bash:
if it failed, abort if the user interrupted).
**For errors:** Also determine:
- `ERROR_CLASS`: a short category — one of: `timeout`, `test_failure`, `build_failure`,
`git_error`, `auth_error`, `network_error`, `browse_error`, `lint_error`,
`merge_conflict`, `permission_error`, `unknown_error`. Pick the most specific match.
- `ERROR_MESSAGE`: a one-line summary of what went wrong (max 200 chars). Include the
command that failed and the key error text. Example: `"bun test: 3 tests failed in
auth.test.ts — expected 200 got 401"`. Never include file paths, secrets, or PII.
- `FAILED_STEP`: which step in the skill workflow failed. Example: `"run_tests"`,
`"create_pr"`, `"merge_base"`, `"build"`, `"qa_browse"`. Use snake_case, max 30 chars.
Run this bash:
```bash
_TEL_END=$(date +%s)
@@ -232,12 +244,16 @@ _TEL_DUR=$(( _TEL_END - _TEL_START ))
rm -f ~/.gstack/analytics/.pending-"$_SESSION_ID" 2>/dev/null || true
~/.codex/skills/gstack/bin/gstack-telemetry-log \
--skill "SKILL_NAME" --duration "$_TEL_DUR" --outcome "OUTCOME" \
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" 2>/dev/null &
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" \
--error-class "ERROR_CLASS" --error-message "ERROR_MESSAGE" \
--failed-step "FAILED_STEP" 2>/dev/null &
```
Replace `SKILL_NAME` with the actual skill name from frontmatter, `OUTCOME` with
success/error/abort, and `USED_BROWSE` with true/false based on whether `$B` was used.
If you cannot determine the outcome, use "unknown". This runs in the background and
For `ERROR_CLASS`, `ERROR_MESSAGE`, and `FAILED_STEP`: use empty string `""` if the
outcome is not error. If the outcome is error but you cannot determine the details,
use `"unknown_error"`, `""`, and `""` respectively. This runs in the background and
never blocks the user.
## Step 0: Detect base branch
+19 -3
View File
@@ -259,7 +259,19 @@ RECOMMENDATION: [what the user should do next]
After the skill workflow completes (success, error, or abort), log the telemetry event.
Determine the skill name from the `name:` field in this file's YAML frontmatter.
Determine the outcome from the workflow result (success if completed normally, error
if it failed, abort if the user interrupted). Run this bash:
if it failed, abort if the user interrupted).
**For errors:** Also determine:
- `ERROR_CLASS`: a short category — one of: `timeout`, `test_failure`, `build_failure`,
`git_error`, `auth_error`, `network_error`, `browse_error`, `lint_error`,
`merge_conflict`, `permission_error`, `unknown_error`. Pick the most specific match.
- `ERROR_MESSAGE`: a one-line summary of what went wrong (max 200 chars). Include the
command that failed and the key error text. Example: `"bun test: 3 tests failed in
auth.test.ts — expected 200 got 401"`. Never include file paths, secrets, or PII.
- `FAILED_STEP`: which step in the skill workflow failed. Example: `"run_tests"`,
`"create_pr"`, `"merge_base"`, `"build"`, `"qa_browse"`. Use snake_case, max 30 chars.
Run this bash:
```bash
_TEL_END=$(date +%s)
@@ -267,12 +279,16 @@ _TEL_DUR=$(( _TEL_END - _TEL_START ))
rm -f ~/.gstack/analytics/.pending-"$_SESSION_ID" 2>/dev/null || true
~/.codex/skills/gstack/bin/gstack-telemetry-log \
--skill "SKILL_NAME" --duration "$_TEL_DUR" --outcome "OUTCOME" \
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" 2>/dev/null &
--used-browse "USED_BROWSE" --session-id "$_SESSION_ID" \
--error-class "ERROR_CLASS" --error-message "ERROR_MESSAGE" \
--failed-step "FAILED_STEP" 2>/dev/null &
```
Replace `SKILL_NAME` with the actual skill name from frontmatter, `OUTCOME` with
success/error/abort, and `USED_BROWSE` with true/false based on whether `$B` was used.
If you cannot determine the outcome, use "unknown". This runs in the background and
For `ERROR_CLASS`, `ERROR_MESSAGE`, and `FAILED_STEP`: use empty string `""` if the
outcome is not error. If the outcome is error but you cannot determine the details,
use `"unknown_error"`, `""`, and `""` respectively. This runs in the background and
never blocks the user.
If `PROACTIVE` is `false`: do NOT proactively suggest other gstack skills during this session.