feat: test bootstrap, regression tests, coverage audit, retro test health

- Add {{TEST_BOOTSTRAP}} resolver to gen-skill-docs.ts
- Add Phase 8e.5 regression test generation to /qa and /qa-design-review
- Add Step 3.4 test coverage audit with quality scoring to /ship
- Add test health tracking to /retro
- Add 2 E2E evals (bootstrap + coverage audit)
- Add 26 validation tests
- Update ARCHITECTURE.md placeholder table
- Add 2 P3 TODOs (CI/CD non-GitHub, auto-upgrade weak tests)
This commit is contained in:
Garry Tan
2026-03-17 10:41:40 -07:00
parent 73b00b4e29
commit 04eb7824bd
16 changed files with 1512 additions and 6 deletions
+28 -1
View File
@@ -164,6 +164,15 @@ cat ~/.gstack/greptile-history.md 2>/dev/null || true
# 9. TODOS.md backlog (if available)
cat TODOS.md 2>/dev/null || true
# 10. Test file count
find . -name '*.test.*' -o -name '*.spec.*' -o -name '*_test.*' -o -name '*_spec.*' 2>/dev/null | grep -v node_modules | wc -l
# 11. Regression test commits in window
git log origin/<default> --since="<window>" --oneline --grep="test(qa):" --grep="test(design):" --grep="test: coverage"
# 12. Test files changed in window
git log origin/<default> --since="<window>" --format="" --name-only | grep -E '\.(test|spec)\.' | sort -u | wc -l
```
### Step 2: Compute Metrics
@@ -185,6 +194,7 @@ Calculate and present these metrics in a summary table:
| Detected sessions | N |
| Avg LOC/session-hour | N |
| Greptile signal | N% (Y catches, Z FPs) |
| Test Health | N total tests · M added this period · K regression tests |
Then show a **per-author leaderboard** immediately below:
@@ -408,7 +418,17 @@ Use the Write tool to save the JSON file with this schema:
}
```
**Note:** Only include the `greptile` field if `~/.gstack/greptile-history.md` exists and has entries within the time window. Only include the `backlog` field if `TODOS.md` exists. If either has no data, omit the field entirely.
**Note:** Only include the `greptile` field if `~/.gstack/greptile-history.md` exists and has entries within the time window. Only include the `backlog` field if `TODOS.md` exists. Only include the `test_health` field if test files were found (command 10 returns > 0). If any has no data, omit the field entirely.
Include test health data in the JSON when test files exist:
```json
"test_health": {
"total_test_files": 47,
"tests_added_this_period": 5,
"regression_test_commits": 3,
"test_files_changed": 8
}
```
Include backlog data in the JSON when TODOS.md exists:
```json
@@ -464,6 +484,13 @@ Narrative covering:
- Any XL PRs that should have been split
- Greptile signal ratio and trend (if history exists): "Greptile: X% signal (Y valid catches, Z false positives)"
### Test Health
- Total test files: N (from command 10)
- Tests added this period: M (from command 12 — test files changed)
- Regression test commits: list `test(qa):` and `test(design):` and `test: coverage` commits from command 11
- If prior retro exists and has `test_health`: show delta "Test count: {last} → {now} (+{delta})"
- If test ratio < 20%: flag as growth area — "100% test coverage is the goal. Tests make vibe coding safe."
### Focus & Highlights
(from Step 8)
- Focus score with interpretation
+28 -1
View File
@@ -99,6 +99,15 @@ cat ~/.gstack/greptile-history.md 2>/dev/null || true
# 9. TODOS.md backlog (if available)
cat TODOS.md 2>/dev/null || true
# 10. Test file count
find . -name '*.test.*' -o -name '*.spec.*' -o -name '*_test.*' -o -name '*_spec.*' 2>/dev/null | grep -v node_modules | wc -l
# 11. Regression test commits in window
git log origin/<default> --since="<window>" --oneline --grep="test(qa):" --grep="test(design):" --grep="test: coverage"
# 12. Test files changed in window
git log origin/<default> --since="<window>" --format="" --name-only | grep -E '\.(test|spec)\.' | sort -u | wc -l
```
### Step 2: Compute Metrics
@@ -120,6 +129,7 @@ Calculate and present these metrics in a summary table:
| Detected sessions | N |
| Avg LOC/session-hour | N |
| Greptile signal | N% (Y catches, Z FPs) |
| Test Health | N total tests · M added this period · K regression tests |
Then show a **per-author leaderboard** immediately below:
@@ -343,7 +353,17 @@ Use the Write tool to save the JSON file with this schema:
}
```
**Note:** Only include the `greptile` field if `~/.gstack/greptile-history.md` exists and has entries within the time window. Only include the `backlog` field if `TODOS.md` exists. If either has no data, omit the field entirely.
**Note:** Only include the `greptile` field if `~/.gstack/greptile-history.md` exists and has entries within the time window. Only include the `backlog` field if `TODOS.md` exists. Only include the `test_health` field if test files were found (command 10 returns > 0). If any has no data, omit the field entirely.
Include test health data in the JSON when test files exist:
```json
"test_health": {
"total_test_files": 47,
"tests_added_this_period": 5,
"regression_test_commits": 3,
"test_files_changed": 8
}
```
Include backlog data in the JSON when TODOS.md exists:
```json
@@ -399,6 +419,13 @@ Narrative covering:
- Any XL PRs that should have been split
- Greptile signal ratio and trend (if history exists): "Greptile: X% signal (Y valid catches, Z false positives)"
### Test Health
- Total test files: N (from command 10)
- Tests added this period: M (from command 12 — test files changed)
- Regression test commits: list `test(qa):` and `test(design):` and `test: coverage` commits from command 11
- If prior retro exists and has `test_health`: show delta "Test count: {last} → {now} (+{delta})"
- If test ratio < 20%: flag as growth area — "100% test coverage is the goal. Tests make vibe coding safe."
### Focus & Highlights
(from Step 8)
- Focus score with interpretation