CalvinBackup/gstack
1
0
Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-09-17 10:25:33 +02:00
Code Issues Packages Projects Releases Wiki Activity

v1.87.4.0 fix: preserve health failures and disclose coverage (#2882)

Browse Source 
* fix: report health failures and coverage accurately

* chore: prepare health reporting release 1.87.4.0

* test: restrict routing evaluations to installed project skills

* test: stabilize health selection and terminal fixtures
This commit is contained in:
Garry Tan
2026-09-15 21:35:11 -07:00
committed by GitHub
parent 85b8c038fc
commit a6b3a57512
15 changed files with 728 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files
health/SKILL.md
+76 -16
View File
@@ -489,22 +489,55 @@ section in CLAUDE.md:
Run each detected tool. For each tool:
1. Record the start time
2. Run the command, capturing both stdout and stderr
3. Record the exit code
2. Run the command, capturing complete stdout and stderr in a private temporary log
3. Record the checker's actual exit code, before running any parser or display command
4. Record the end time
5. Capture the last 50 lines of output for the report
5. Parse counts from the complete log, then display its last 50 lines for the report
```bash
# Example for each tool — run each independently
START=$(date +%s)
tsc --noEmit 2>&1 | tail -50
EXIT_CODE=$?
END=$(date +%s)
echo "TOOL:typecheck EXIT:$EXIT_CODE DURATION:$((END-START))s"
# Capture example — run each tool independently; adapt the command and parser.
(
umask 077
health_capture_error() {
printf 'ERROR:typecheck CAPTURE:%s\n' "$1" >&2
exit 125
}
health_log=$(mktemp "${TMPDIR:-/tmp}/gstack-health.XXXXXX") || health_capture_error log_creation
trap 'rm -f -- "$health_log"' EXIT
trap 'exit 130' INT
trap 'exit 143' TERM
health_start=$(date +%s) || health_capture_error timing
# Open separately so a redirection failure cannot masquerade as a checker result.
if ! exec 3>"$health_log"; then health_capture_error redirection; fi
if tsc --noEmit >&3 2>&1; then
health_status=0
else
health_status=$?
fi
exec 3>&-
health_end=$(date +%s) || health_capture_error timing
# awk returns a successful zero count for no matches, including empty output.
health_count=$(awk '/error TS/ { count++ } END { print count+0 }' "$health_log") || health_capture_error parsing
tail -50 "$health_log" || health_capture_error display
printf 'TOOL:typecheck EXIT:%s DURATION:%ss ERRORS:%s\n' "$health_status" "$((health_end-health_start))" "$health_count"
exit "$health_status"
)
```
Run tools sequentially (some may share resources or lock files). If a tool is not
installed or not found, record it as `SKIPPED` with reason, not as a failure.
Run tools sequentially in independent invocations (some may share resources or lock
files). A failed checker must not prevent later tools from running. Remember each
reported exit code and full-log counts; never use the status of `tail` or a parser
as the checker result. Guard parsers whose no-match exit is expected.
Before running a tool, check availability using the project's configured command
and local tool installation. If availability detection establishes that it is
missing, record `SKIPPED` with the reason. An executed checker returning 127 is a
failure, not evidence that the category should be skipped.
Capture failures (log creation, redirection, parsing, or display) are `ERROR`, never
`CLEAN` or `SKIPPED`. Include the cause and do not invent a category score. Report
the composite as `N/A — capture failed` if any category cannot be scored for this
reason; do not redistribute that category's weight or persist a numeric history row.
---
@@ -522,6 +555,9 @@ Score each category on a 0-10 scale using this rubric:
| GBrain (D6) | 10% | doctor=ok, queue<10, pushed <24h | doctor=warnings OR queue<100 OR pushed <72h | doctor broken OR queue>=100 OR pushed >=72h | N/A (gbrain not installed) |
**Parsing tool output for counts:**
Use the complete captured output, not the displayed tail. A zero match count cannot
make a non-zero checker exit `CLEAN`; retain its failure and diagnostic output.
- **tsc:** Count lines matching `error TS` in output.
- **biome/eslint/ruff:** Count lines matching error/warning patterns. Parse the summary line if available.
- **Tests:** Parse pass/fail counts from the test runner output. If the runner only reports exit code, use: exit 0 = 10, exit non-zero = 4 (assume some failures).
@@ -537,6 +573,11 @@ If a category is skipped (tool not available — includes GBrain when gbrain
is not installed), redistribute its weight proportionally among the
remaining categories.
Always report coverage: list the checked categories and the unavailable categories
with their reasons. Label a numeric composite with skipped categories as **partial
coverage**. If zero checks executed, report **N/A — no checks ran**, do not compute
a numeric composite, and skip numeric history persistence and trend calculation.
**GBrain sub-score computation (D6):**
```
@@ -579,6 +620,9 @@ Shell lint shellcheck 10/10 CLEAN 1s 0 issues
GBrain gbrain doctor 10/10 CLEAN <1s doctor=ok, queue=3, pushed 2h ago
COMPOSITE SCORE: 9.1 / 10
Coverage: 6/6 categories checked
Checked: typecheck, lint, test, deadcode, shell, gbrain
Unavailable: none
Duration: 23s total
```
@@ -588,6 +632,13 @@ Use these status labels:
- 7-9: `WARNING`
- 4-6: `NEEDS WORK`
- 0-3: `CRITICAL`
- Unavailable tool: `SKIPPED` (no score)
- Capture failure: `ERROR` (no score; composite is N/A)
For partial coverage, show e.g. `COMPOSITE SCORE: 8.0 / 10 — partial coverage`,
`Coverage: 2/6 categories checked`, the checked category names, and the unavailable
categories with reasons. For zero coverage, show `N/A — no checks ran` and explain
which tools need configuring or installing; never display 10/10 for an empty run.
If any category scored below 7, list the top issues from that tool's output:
@@ -607,7 +658,9 @@ DETAILS: Lint (3 warnings)
eval "$(~/.claude/skills/gstack/bin/gstack-slug 2>/dev/null)" && mkdir -p ~/.gstack/projects/$SLUG
```
Append one JSONL line to `~/.gstack/projects/$SLUG/health-history.jsonl`:
Only when a numeric composite exists, append one JSONL line to
`~/.gstack/projects/$SLUG/health-history.jsonl`. Zero-check and capture-error runs
must leave any existing history unchanged:
```json
{"ts":"2026-03-31T14:30:00Z","branch":"main","score":9.1,"typecheck":10,"lint":8,"test":10,"deadcode":7,"shell":10,"gbrain":10,"duration_s":23}
@@ -636,7 +689,14 @@ eval "$(~/.claude/skills/gstack/bin/gstack-slug 2>/dev/null)" && mkdir -p ~/.gst
tail -10 ~/.gstack/projects/$SLUG/health-history.jsonl 2>/dev/null || echo "NO_HISTORY"
```
**If prior entries exist, show the trend:**
**Compare like-for-like coverage.** For each history row, form the set of categories
with non-null scores (missing fields count as null). Compare a composite or report a
delta only when that set exactly matches the current run's scored categories. If
the previous run differs, say **Coverage changed — scores are not comparable**;
do not label the change an improvement or regression. Historical rows may still be
shown with their unavailable categories marked. A current N/A result has no trend.
**If comparable prior entries exist, show the trend:**
```
HEALTH TREND (last 5 runs)
@@ -651,7 +711,7 @@ Date Branch Score TC Lint Test Dead Shell GBrain
Trend: IMPROVING (+0.9 since last run)
```
**If score dropped vs the previous run:**
**If score dropped vs the previous run with identical coverage:**
1. Identify WHICH categories declined
2. Show the delta for each declining category
3. Correlate with tool output -- what specific errors/warnings appeared?
@@ -689,7 +749,7 @@ Rank by `weight * (10 - score)` descending. Only show categories below 10.
1. **Wrap, don't replace.** Run the project's own tools. Never substitute your own analysis for what the tool reports.
2. **Read-only.** Never fix issues. Present the dashboard and let the user decide.
3. **Respect CLAUDE.md.** If `## Health Stack` is configured, use those exact commands. Do not second-guess.
4. **Skipped is not failed.** If a tool isn't available, skip it gracefully and redistribute weight. Do not penalize the score.
4. **Skipped is not failed.** Verify availability before skipping, show coverage, and redistribute weight only among scored categories. An executed command's failure must not become a skip.
5. **Show raw output for failures.** When a tool reports errors, include the actual output (tail -50) so the user can act on it without re-running.
6. **Trends require history.** On first run, say "First health check -- no trend data yet. Run /health again after making changes to track progress."
6. **Trends require comparable history.** On the first scored run, say "First health check -- no trend data yet. Run /health again after making changes to track progress." Changed coverage and N/A runs have no score delta.
7. **Be honest about scores.** A codebase with 100 type errors and all tests passing is not healthy. The composite score should reflect reality.
health/SKILL.md.tmpl
+76 -16
View File
@@ -114,22 +114,55 @@ section in CLAUDE.md:
Run each detected tool. For each tool:
1. Record the start time
2. Run the command, capturing both stdout and stderr
3. Record the exit code
2. Run the command, capturing complete stdout and stderr in a private temporary log
3. Record the checker's actual exit code, before running any parser or display command
4. Record the end time
5. Capture the last 50 lines of output for the report
5. Parse counts from the complete log, then display its last 50 lines for the report
```bash
# Example for each tool — run each independently
START=$(date +%s)
tsc --noEmit 2>&1 | tail -50
EXIT_CODE=$?
END=$(date +%s)
echo "TOOL:typecheck EXIT:$EXIT_CODE DURATION:$((END-START))s"
# Capture example — run each tool independently; adapt the command and parser.
(
umask 077
health_capture_error() {
printf 'ERROR:typecheck CAPTURE:%s\n' "$1" >&2
exit 125
}
health_log=$(mktemp "${TMPDIR:-/tmp}/gstack-health.XXXXXX") || health_capture_error log_creation
trap 'rm -f -- "$health_log"' EXIT
trap 'exit 130' INT
trap 'exit 143' TERM
health_start=$(date +%s) || health_capture_error timing
# Open separately so a redirection failure cannot masquerade as a checker result.
if ! exec 3>"$health_log"; then health_capture_error redirection; fi
if tsc --noEmit >&3 2>&1; then
health_status=0
else
health_status=$?
fi
exec 3>&-
health_end=$(date +%s) || health_capture_error timing
# awk returns a successful zero count for no matches, including empty output.
health_count=$(awk '/error TS/ { count++ } END { print count+0 }' "$health_log") || health_capture_error parsing
tail -50 "$health_log" || health_capture_error display
printf 'TOOL:typecheck EXIT:%s DURATION:%ss ERRORS:%s\n' "$health_status" "$((health_end-health_start))" "$health_count"
exit "$health_status"
)
```
Run tools sequentially (some may share resources or lock files). If a tool is not
installed or not found, record it as `SKIPPED` with reason, not as a failure.
Run tools sequentially in independent invocations (some may share resources or lock
files). A failed checker must not prevent later tools from running. Remember each
reported exit code and full-log counts; never use the status of `tail` or a parser
as the checker result. Guard parsers whose no-match exit is expected.
Before running a tool, check availability using the project's configured command
and local tool installation. If availability detection establishes that it is
missing, record `SKIPPED` with the reason. An executed checker returning 127 is a
failure, not evidence that the category should be skipped.
Capture failures (log creation, redirection, parsing, or display) are `ERROR`, never
`CLEAN` or `SKIPPED`. Include the cause and do not invent a category score. Report
the composite as `N/A — capture failed` if any category cannot be scored for this
reason; do not redistribute that category's weight or persist a numeric history row.
---
@@ -147,6 +180,9 @@ Score each category on a 0-10 scale using this rubric:
| GBrain (D6) | 10% | doctor=ok, queue<10, pushed <24h | doctor=warnings OR queue<100 OR pushed <72h | doctor broken OR queue>=100 OR pushed >=72h | N/A (gbrain not installed) |
**Parsing tool output for counts:**
Use the complete captured output, not the displayed tail. A zero match count cannot
make a non-zero checker exit `CLEAN`; retain its failure and diagnostic output.
- **tsc:** Count lines matching `error TS` in output.
- **biome/eslint/ruff:** Count lines matching error/warning patterns. Parse the summary line if available.
- **Tests:** Parse pass/fail counts from the test runner output. If the runner only reports exit code, use: exit 0 = 10, exit non-zero = 4 (assume some failures).
@@ -162,6 +198,11 @@ If a category is skipped (tool not available — includes GBrain when gbrain
is not installed), redistribute its weight proportionally among the
remaining categories.
Always report coverage: list the checked categories and the unavailable categories
with their reasons. Label a numeric composite with skipped categories as **partial
coverage**. If zero checks executed, report **N/A — no checks ran**, do not compute
a numeric composite, and skip numeric history persistence and trend calculation.
**GBrain sub-score computation (D6):**
```
@@ -204,6 +245,9 @@ Shell lint shellcheck 10/10 CLEAN 1s 0 issues
GBrain gbrain doctor 10/10 CLEAN <1s doctor=ok, queue=3, pushed 2h ago
COMPOSITE SCORE: 9.1 / 10
Coverage: 6/6 categories checked
Checked: typecheck, lint, test, deadcode, shell, gbrain
Unavailable: none
Duration: 23s total
```
@@ -213,6 +257,13 @@ Use these status labels:
- 7-9: `WARNING`
- 4-6: `NEEDS WORK`
- 0-3: `CRITICAL`
- Unavailable tool: `SKIPPED` (no score)
- Capture failure: `ERROR` (no score; composite is N/A)
For partial coverage, show e.g. `COMPOSITE SCORE: 8.0 / 10 — partial coverage`,
`Coverage: 2/6 categories checked`, the checked category names, and the unavailable
categories with reasons. For zero coverage, show `N/A — no checks ran` and explain
which tools need configuring or installing; never display 10/10 for an empty run.
If any category scored below 7, list the top issues from that tool's output:
@@ -232,7 +283,9 @@ DETAILS: Lint (3 warnings)
{{SLUG_SETUP}}
```
Append one JSONL line to `~/.gstack/projects/$SLUG/health-history.jsonl`:
Only when a numeric composite exists, append one JSONL line to
`~/.gstack/projects/$SLUG/health-history.jsonl`. Zero-check and capture-error runs
must leave any existing history unchanged:
```json
{"ts":"2026-03-31T14:30:00Z","branch":"main","score":9.1,"typecheck":10,"lint":8,"test":10,"deadcode":7,"shell":10,"gbrain":10,"duration_s":23}
@@ -261,7 +314,14 @@ file exists and has prior entries).
tail -10 ~/.gstack/projects/$SLUG/health-history.jsonl 2>/dev/null || echo "NO_HISTORY"
```
**If prior entries exist, show the trend:**
**Compare like-for-like coverage.** For each history row, form the set of categories
with non-null scores (missing fields count as null). Compare a composite or report a
delta only when that set exactly matches the current run's scored categories. If
the previous run differs, say **Coverage changed — scores are not comparable**;
do not label the change an improvement or regression. Historical rows may still be
shown with their unavailable categories marked. A current N/A result has no trend.
**If comparable prior entries exist, show the trend:**
```
HEALTH TREND (last 5 runs)
@@ -276,7 +336,7 @@ Date Branch Score TC Lint Test Dead Shell GBrain
Trend: IMPROVING (+0.9 since last run)
```
**If score dropped vs the previous run:**
**If score dropped vs the previous run with identical coverage:**
1. Identify WHICH categories declined
2. Show the delta for each declining category
3. Correlate with tool output -- what specific errors/warnings appeared?
@@ -314,7 +374,7 @@ Rank by `weight * (10 - score)` descending. Only show categories below 10.
1. **Wrap, don't replace.** Run the project's own tools. Never substitute your own analysis for what the tool reports.
2. **Read-only.** Never fix issues. Present the dashboard and let the user decide.
3. **Respect CLAUDE.md.** If `## Health Stack` is configured, use those exact commands. Do not second-guess.
4. **Skipped is not failed.** If a tool isn't available, skip it gracefully and redistribute weight. Do not penalize the score.
4. **Skipped is not failed.** Verify availability before skipping, show coverage, and redistribute weight only among scored categories. An executed command's failure must not become a skip.
5. **Show raw output for failures.** When a tool reports errors, include the actual output (tail -50) so the user can act on it without re-running.
6. **Trends require history.** On first run, say "First health check -- no trend data yet. Run /health again after making changes to track progress."
6. **Trends require comparable history.** On the first scored run, say "First health check -- no trend data yet. Run /health again after making changes to track progress." Changed coverage and N/A runs have no score delta.
7. **Be honest about scores.** A codebase with 100 type errors and all tests passing is not healthy. The composite score should reflect reality.