feat: test coverage catalog + failure triage (merged branches) (#285)

* feat: add bin/gstack-repo-mode — solo vs collaborative detection with caching

Detects whether a repo is solo-dev (one person does 80%+ of recent commits)
or collaborative. Uses 90-day git shortlog window with 7-day cache in
~/.gstack/projects/{SLUG}/repo-mode.json. Config override via
`gstack-config set repo_mode solo|collaborative` takes precedence over
the heuristic. Minimum 5 commits required to classify (otherwise unknown).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: test failure ownership triage — see something say something

Adds two new preamble sections to all gstack skills:
- Repo Ownership Mode: explains solo vs collaborative behavior
- See Something, Say Something: proactive issue flagging principle

Adds {{TEST_FAILURE_TRIAGE}} template variable (opt-in, used by /ship):
- Classifies test failures as in-branch vs pre-existing
- Solo mode defaults to "investigate and fix now"
- Collaborative mode offers "blame + assign GitHub issue" option
- Also offers P0 TODO and skip options

/ship Step 3 now triages test failures instead of hard-stopping on all
failures. In-branch failures still block shipping. Pre-existing failures
get user-directed triage based on repo mode.

Adds P2 TODO for gstack notes system (deferred lightweight reminder).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate SKILL.md files for Claude and Codex hosts

All 22 Claude skills and 21 Codex skills regenerated with new preamble
sections (Repo Ownership Mode, See Something Say Something) and
{{TEST_FAILURE_TRIAGE}} resolved in ship/SKILL.md.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: validate repo mode values to prevent shell injection

Codex adversarial review found that unvalidated config/cache values
could be injected into shell via source <(gstack-repo-mode). Added
validate_mode() that only allows solo|collaborative|unknown — anything
else becomes "unknown". Prevents persistent code execution through
malicious config.yaml or tampered cache JSON.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: shell injection via branch names + feature-branch sampling bias

Codex code review found two issues:

P1: eval $(gstack-slug) in gstack-repo-mode executes branch names as
shell. Branch names like foo$(touch${IFS}pwned) are valid git refs and
would execute arbitrary commands. Fix: compute SLUG directly with sed
instead of eval'ing gstack-slug output.

P2: git shortlog HEAD only sees current branch history. On feature
branches that haven't merged main recently, other contributors disappear
from the sample. Fix: use git shortlog on the default branch
(origin/main) instead of HEAD.

Also improved blame lookup in collaborative triage to check both the
test file and the production code it covers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: broaden codex-host stripping test to accommodate triage section

"Investigate and fix" now appears in TEST_FAILURE_TRIAGE (not just the
Codex review step). Use CODEX_REVIEWS config string as a more specific
marker for detecting the Codex review step in Codex-hosted skills.

* fix: replace template placeholder in TODOS.md with readable text

{{TEST_FAILURE_TRIAGE}} is template syntax but TODOS.md is not processed
by gen-skill-docs — replaced with human-readable reference.

* chore: bump version and changelog (v0.9.5.0)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add bin/ directory to project structure in CLAUDE.md

* test: add triage resolver unit tests, plan-eng coverage audit E2E, and triage E2E

- TEST_FAILURE_TRIAGE resolver: 6 unit tests verifying all triage steps (T1-T4),
  REPO_MODE branching, and safety default for ambiguous failures
- plan-eng-coverage-audit E2E: tests /plan-eng-review coverage audit codepath
  (gap identified during eng review — existed on neither branch)
- ship-triage E2E: planted-bug fixture with in-branch (truncate null) and
  pre-existing (divide-by-zero) failures; verifies correct classification
- Touchfile entries for diff-based test selection

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate stale Codex SKILL.md for retro

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-03-21 09:36:17 -07:00
committed by GitHub
parent 624c7793de
commit f954f3449a
43 changed files with 1232 additions and 18 deletions
+11 -1
View File
@@ -1,6 +1,6 @@
# Changelog
## [0.9.5.0] - 2026-03-20 — Test Coverage Catalog
## [0.9.5.0] - 2026-03-20 — Test Coverage Catalog + See Something, Say Something
### Added
@@ -9,6 +9,16 @@
- **E2E test recommendations built in.** The coverage audit now knows when to recommend E2E tests (common user flows, tricky integrations) vs unit tests, and flags LLM prompt changes that need eval coverage.
- **Regression detection iron rule.** When a code change modifies existing behavior, gstack now always generates a regression test — no asking, no skipping.
- **Test framework auto-detection.** Reads your CLAUDE.md for test commands first, then auto-detects from project files. Works with any framework.
- **Solo dev repos get test failures fixed, not just flagged.** gstack now detects whether you're a solo dev (80%+ of commits) or on a team, and adapts its behavior. Solo devs get offered "investigate and fix now" for pre-existing test failures — because you're the only one who will. Team repos get "blame + assign GitHub issue" to route problems to the right person.
- **Pre-existing test failures no longer block shipping.** When `/ship` hits a test failure that wasn't caused by your branch, gstack classifies it (in-branch vs pre-existing) and offers options: fix now, add as P0 TODO, assign to the author, or skip. Only in-branch failures block the workflow.
- **`bin/gstack-repo-mode` detects solo vs collaborative repos.** Uses 90-day git history with an 80% threshold, 7-day cache, and config override (`gstack-config set repo_mode solo`). Every skill's preamble now outputs `REPO_MODE`.
- **"See Something, Say Something" is now a universal principle.** Every gstack skill notices issues outside your branch — deprecation warnings, dead code, env problems — and flags them. In solo mode, it offers to fix. In collaborative mode, it flags and moves on.
### Fixed
- **Shell injection via branch names prevented.** `gstack-repo-mode` computes the project slug directly from `git remote` instead of eval'ing `gstack-slug`, which could execute shell metacharacters in branch names.
- **Feature-branch sampling bias eliminated.** Repo mode detection now uses the default branch (`origin/main`) for git history, not `HEAD`, so feature branches with unusual commit patterns don't skew the solo/collaborative classification.
- **Repo mode config values validated.** Only `solo`, `collaborative`, and `unknown` are accepted — arbitrary strings can't be injected via `source <(...)`.
## [0.9.4.1] - 2026-03-20