mirror of
https://github.com/garrytan/gstack.git
synced 2026-05-01 19:25:10 +02:00
7ff0f84b1e
* refactor: extract {{TEST_COVERAGE_AUDIT}} shared resolver
DRY extraction of the test coverage audit methodology into a shared
generator function with three explicit placeholders:
- TEST_COVERAGE_AUDIT_PLAN (plan-eng-review)
- TEST_COVERAGE_AUDIT_SHIP (ship)
- TEST_COVERAGE_AUDIT_REVIEW (review)
Shared across all modes: codepath tracing, ASCII diagram format,
quality scoring rubric, E2E test decision matrix, regression rule,
and test framework detection via CLAUDE.md.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: plan-eng-review uses shared test coverage audit
Replace the thin 6-line Section 3 test review with the full shared
methodology via {{TEST_COVERAGE_AUDIT_PLAN}}. Plan mode now:
- Traces every codepath with full ASCII diagrams
- Adds missing tests to the plan (not just "check for tests")
- Writes test plan artifact for /qa consumption
- Includes E2E/eval recommendations and regression detection
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: ship uses shared test coverage audit
Replace 135 lines of inline Step 3.4 methodology with
{{TEST_COVERAGE_AUDIT_SHIP}}. Functionally identical output plus:
- E2E test decision matrix (marks paths needing E2E vs unit)
- Eval recommendations for LLM prompt changes
- Regression detection iron rule
- Test framework detection via CLAUDE.md first
- Test plan artifact for /qa consumption
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: /review Step 4.75 test coverage diagram
Add codepath tracing to the pre-landing review via
{{TEST_COVERAGE_AUDIT_REVIEW}}. Review mode:
- Produces ASCII coverage diagram (same methodology as plan/ship)
- Generates tests for gaps via Fix-First (ASK user)
- Subsumes Pass 2 "Test Gaps" checklist category
- Gaps are INFORMATIONAL findings
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: mode differentiation + regression guard for coverage audit
10 new tests verifying the three TEST_COVERAGE_AUDIT placeholders:
- All modes share: codepath tracing, E2E matrix, regression rule
- Plan mode: adds to plan + artifact, no ship-specific content
- Ship mode: auto-generates + before/after count + coverage summary
- Review mode: Fix-First ASK + INFORMATIONAL, no artifact
- Regression guard: ship SKILL.md preserves all key phrases
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: extract shared coverage audit fixture + review E2E
- Extract billing.ts fixture into coverage-audit-fixture.ts (DRY)
- Refactor ship-coverage-audit E2E to use shared fixture
- Add review-coverage-audit E2E for Step 4.75
- Update touchfiles: both E2Es depend on shared fixture
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: strengthen E2E assertions for coverage audit tests
The coverage audit E2E tests (ship + review) were only asserting
exitReason === 'success' and readCalls > 0 — they passed even
if the agent produced no coverage diagram. Add assertion that
the output contains either GAP or TESTED markers.
Found during /review.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: plan mode traces the plan, not the git diff
Codex adversarial review caught that plan-eng-review was inheriting
"git diff origin/<base>...HEAD" from the shared resolver, but plan mode
reviews a plan document, not a code diff. Plan mode now says:
"Trace every codepath in the plan" and "Read the plan document."
Ship and review modes keep the git diff instruction.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: bump version and changelog (v0.9.5.0)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: test coverage catalog + failure triage (merged branches) (#285)
* feat: add bin/gstack-repo-mode — solo vs collaborative detection with caching
Detects whether a repo is solo-dev (one person does 80%+ of recent commits)
or collaborative. Uses 90-day git shortlog window with 7-day cache in
~/.gstack/projects/{SLUG}/repo-mode.json. Config override via
`gstack-config set repo_mode solo|collaborative` takes precedence over
the heuristic. Minimum 5 commits required to classify (otherwise unknown).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: test failure ownership triage — see something say something
Adds two new preamble sections to all gstack skills:
- Repo Ownership Mode: explains solo vs collaborative behavior
- See Something, Say Something: proactive issue flagging principle
Adds {{TEST_FAILURE_TRIAGE}} template variable (opt-in, used by /ship):
- Classifies test failures as in-branch vs pre-existing
- Solo mode defaults to "investigate and fix now"
- Collaborative mode offers "blame + assign GitHub issue" option
- Also offers P0 TODO and skip options
/ship Step 3 now triages test failures instead of hard-stopping on all
failures. In-branch failures still block shipping. Pre-existing failures
get user-directed triage based on repo mode.
Adds P2 TODO for gstack notes system (deferred lightweight reminder).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: regenerate SKILL.md files for Claude and Codex hosts
All 22 Claude skills and 21 Codex skills regenerated with new preamble
sections (Repo Ownership Mode, See Something Say Something) and
{{TEST_FAILURE_TRIAGE}} resolved in ship/SKILL.md.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: validate repo mode values to prevent shell injection
Codex adversarial review found that unvalidated config/cache values
could be injected into shell via source <(gstack-repo-mode). Added
validate_mode() that only allows solo|collaborative|unknown — anything
else becomes "unknown". Prevents persistent code execution through
malicious config.yaml or tampered cache JSON.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: shell injection via branch names + feature-branch sampling bias
Codex code review found two issues:
P1: eval $(gstack-slug) in gstack-repo-mode executes branch names as
shell. Branch names like foo$(touch${IFS}pwned) are valid git refs and
would execute arbitrary commands. Fix: compute SLUG directly with sed
instead of eval'ing gstack-slug output.
P2: git shortlog HEAD only sees current branch history. On feature
branches that haven't merged main recently, other contributors disappear
from the sample. Fix: use git shortlog on the default branch
(origin/main) instead of HEAD.
Also improved blame lookup in collaborative triage to check both the
test file and the production code it covers.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: broaden codex-host stripping test to accommodate triage section
"Investigate and fix" now appears in TEST_FAILURE_TRIAGE (not just the
Codex review step). Use CODEX_REVIEWS config string as a more specific
marker for detecting the Codex review step in Codex-hosted skills.
* fix: replace template placeholder in TODOS.md with readable text
{{TEST_FAILURE_TRIAGE}} is template syntax but TODOS.md is not processed
by gen-skill-docs — replaced with human-readable reference.
* chore: bump version and changelog (v0.9.5.0)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: add bin/ directory to project structure in CLAUDE.md
* test: add triage resolver unit tests, plan-eng coverage audit E2E, and triage E2E
- TEST_FAILURE_TRIAGE resolver: 6 unit tests verifying all triage steps (T1-T4),
REPO_MODE branching, and safety default for ambiguous failures
- plan-eng-coverage-audit E2E: tests /plan-eng-review coverage audit codepath
(gap identified during eng review — existed on neither branch)
- ship-triage E2E: planted-bug fixture with in-branch (truncate null) and
pre-existing (divide-by-zero) failures; verifies correct classification
- Touchfile entries for diff-based test selection
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: regenerate stale Codex SKILL.md for retro
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: gstack-repo-mode handles repos without origin remote
Split `git remote get-url origin` into a separate variable with `|| true`
so the script doesn't crash under `set -euo pipefail` in local-only repos.
Falls back to REPO_MODE=unknown gracefully.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: REPO_MODE defaults to unknown when helper emits nothing
Changed preamble from `source <(...) || REPO_MODE=unknown` (which doesn't
catch empty output) to `source <(...) || true` followed by
`REPO_MODE=${REPO_MODE:-unknown}`. Regenerated all SKILL.md files.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: triage E2E runs both test files in subprocesses
math.test.js called process.exit(1) which killed the runner before
string.test.js could execute. Changed test runner to use child_process
so each test runs independently and both failure classes are exercised.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: gstack-repo-mode handles repos without origin remote
Fall back through origin/main → origin/master → HEAD when
git symbolic-ref refs/remotes/origin/HEAD is not set. Prevents
shortlog crash in repos where origin/HEAD isn't configured.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: triage E2E runs both test files in subprocesses
Add assertions verifying both math.test.js (pre-existing failure) and
string.test.js (in-branch failure) actually executed during triage.
Prevents false passes where only one failure class is exercised.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: REPO_MODE defaults to unknown when helper emits nothing
- Remove head -20 truncation that biased solo classification by
dropping low-volume contributors from the denominator
- Use atomic write (mktemp + mv) for cache to prevent concurrent
preamble reads from seeing partial JSON
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add test coverage catalog to CHANGELOG + update project structure
- CHANGELOG: add 6 entries for coverage audit, review Step 4.75, E2E
recommendations, regression iron rule, failure triage, repo-mode fix
- CLAUDE.md: add missing skill directories (autoplan, benchmark, canary,
codex, land-and-deploy, setup-deploy) to project structure
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: bump version and changelog (v0.10.1.0)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: CHANGELOG rules — branch-scoped versions, never fold into old entries
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
260 lines
11 KiB
Cheetah
260 lines
11 KiB
Cheetah
---
|
|
name: review
|
|
version: 1.0.0
|
|
description: |
|
|
Pre-landing PR review. Analyzes diff against the base branch for SQL safety, LLM trust
|
|
boundary violations, conditional side effects, and other structural issues. Use when
|
|
asked to "review this PR", "code review", "pre-landing review", or "check my diff".
|
|
Proactively suggest when the user is about to merge or land code changes.
|
|
allowed-tools:
|
|
- Bash
|
|
- Read
|
|
- Edit
|
|
- Write
|
|
- Grep
|
|
- Glob
|
|
- Agent
|
|
- AskUserQuestion
|
|
- WebSearch
|
|
---
|
|
|
|
{{PREAMBLE}}
|
|
|
|
{{BASE_BRANCH_DETECT}}
|
|
|
|
# Pre-Landing PR Review
|
|
|
|
You are running the `/review` workflow. Analyze the current branch's diff against the base branch for structural issues that tests don't catch.
|
|
|
|
---
|
|
|
|
## Step 1: Check branch
|
|
|
|
1. Run `git branch --show-current` to get the current branch.
|
|
2. If on the base branch, output: **"Nothing to review — you're on the base branch or have no changes against it."** and stop.
|
|
3. Run `git fetch origin <base> --quiet && git diff origin/<base> --stat` to check if there's a diff. If no diff, output the same message and stop.
|
|
|
|
---
|
|
|
|
## Step 1.5: Scope Drift Detection
|
|
|
|
Before reviewing code quality, check: **did they build what was requested — nothing more, nothing less?**
|
|
|
|
1. Read `TODOS.md` (if it exists). Read PR description (`gh pr view --json body --jq .body 2>/dev/null || true`).
|
|
Read commit messages (`git log origin/<base>..HEAD --oneline`).
|
|
**If no PR exists:** rely on commit messages and TODOS.md for stated intent — this is the common case since /review runs before /ship creates the PR.
|
|
2. Identify the **stated intent** — what was this branch supposed to accomplish?
|
|
3. Run `git diff origin/<base> --stat` and compare the files changed against the stated intent.
|
|
4. Evaluate with skepticism:
|
|
|
|
**SCOPE CREEP detection:**
|
|
- Files changed that are unrelated to the stated intent
|
|
- New features or refactors not mentioned in the plan
|
|
- "While I was in there..." changes that expand blast radius
|
|
|
|
**MISSING REQUIREMENTS detection:**
|
|
- Requirements from TODOS.md/PR description not addressed in the diff
|
|
- Test coverage gaps for stated requirements
|
|
- Partial implementations (started but not finished)
|
|
|
|
5. Output (before the main review begins):
|
|
```
|
|
Scope Check: [CLEAN / DRIFT DETECTED / REQUIREMENTS MISSING]
|
|
Intent: <1-line summary of what was requested>
|
|
Delivered: <1-line summary of what the diff actually does>
|
|
[If drift: list each out-of-scope change]
|
|
[If missing: list each unaddressed requirement]
|
|
```
|
|
|
|
6. This is **INFORMATIONAL** — does not block the review. Proceed to Step 2.
|
|
|
|
---
|
|
|
|
## Step 2: Read the checklist
|
|
|
|
Read `.claude/skills/review/checklist.md`.
|
|
|
|
**If the file cannot be read, STOP and report the error.** Do not proceed without the checklist.
|
|
|
|
---
|
|
|
|
## Step 2.5: Check for Greptile review comments
|
|
|
|
Read `.claude/skills/review/greptile-triage.md` and follow the fetch, filter, classify, and **escalation detection** steps.
|
|
|
|
**If no PR exists, `gh` fails, API returns an error, or there are zero Greptile comments:** Skip this step silently. Greptile integration is additive — the review works without it.
|
|
|
|
**If Greptile comments are found:** Store the classifications (VALID & ACTIONABLE, VALID BUT ALREADY FIXED, FALSE POSITIVE, SUPPRESSED) — you will need them in Step 5.
|
|
|
|
---
|
|
|
|
## Step 3: Get the diff
|
|
|
|
Fetch the latest base branch to avoid false positives from stale local state:
|
|
|
|
```bash
|
|
git fetch origin <base> --quiet
|
|
```
|
|
|
|
Run `git diff origin/<base>` to get the full diff. This includes both committed and uncommitted changes against the latest base branch.
|
|
|
|
---
|
|
|
|
## Step 4: Two-pass review
|
|
|
|
Apply the checklist against the diff in two passes:
|
|
|
|
1. **Pass 1 (CRITICAL):** SQL & Data Safety, Race Conditions & Concurrency, LLM Output Trust Boundary, Enum & Value Completeness
|
|
2. **Pass 2 (INFORMATIONAL):** Conditional Side Effects, Magic Numbers & String Coupling, Dead Code & Consistency, LLM Prompt Issues, Test Gaps, View/Frontend, Performance & Bundle Impact
|
|
|
|
**Enum & Value Completeness requires reading code OUTSIDE the diff.** When the diff introduces a new enum value, status, tier, or type constant, use Grep to find all files that reference sibling values, then Read those files to check if the new value is handled. This is the one category where within-diff review is insufficient.
|
|
|
|
**Search-before-recommending:** When recommending a fix pattern (especially for concurrency, caching, auth, or framework-specific behavior):
|
|
- Verify the pattern is current best practice for the framework version in use
|
|
- Check if a built-in solution exists in newer versions before recommending a workaround
|
|
- Verify API signatures against current docs (APIs change between versions)
|
|
|
|
Takes seconds, prevents recommending outdated patterns. If WebSearch is unavailable, note it and proceed with in-distribution knowledge.
|
|
|
|
Follow the output format specified in the checklist. Respect the suppressions — do NOT flag items listed in the "DO NOT flag" section.
|
|
|
|
---
|
|
|
|
## Step 4.5: Design Review (conditional)
|
|
|
|
{{DESIGN_REVIEW_LITE}}
|
|
|
|
Include any design findings alongside the findings from Step 4. They follow the same Fix-First flow in Step 5 — AUTO-FIX for mechanical CSS fixes, ASK for everything else.
|
|
|
|
---
|
|
|
|
## Step 4.75: Test Coverage Diagram
|
|
|
|
{{TEST_COVERAGE_AUDIT_REVIEW}}
|
|
|
|
This step subsumes the "Test Gaps" category from Pass 2 — do not duplicate findings between the checklist Test Gaps item and this coverage diagram. Include any coverage gaps alongside the findings from Step 4 and Step 4.5. They follow the same Fix-First flow — gaps are INFORMATIONAL findings.
|
|
|
|
---
|
|
|
|
## Step 5: Fix-First Review
|
|
|
|
**Every finding gets action — not just critical ones.**
|
|
|
|
Output a summary header: `Pre-Landing Review: N issues (X critical, Y informational)`
|
|
|
|
### Step 5a: Classify each finding
|
|
|
|
For each finding, classify as AUTO-FIX or ASK per the Fix-First Heuristic in
|
|
checklist.md. Critical findings lean toward ASK; informational findings lean
|
|
toward AUTO-FIX.
|
|
|
|
### Step 5b: Auto-fix all AUTO-FIX items
|
|
|
|
Apply each fix directly. For each one, output a one-line summary:
|
|
`[AUTO-FIXED] [file:line] Problem → what you did`
|
|
|
|
### Step 5c: Batch-ask about ASK items
|
|
|
|
If there are ASK items remaining, present them in ONE AskUserQuestion:
|
|
|
|
- List each item with a number, the severity label, the problem, and a recommended fix
|
|
- For each item, provide options: A) Fix as recommended, B) Skip
|
|
- Include an overall RECOMMENDATION
|
|
|
|
Example format:
|
|
```
|
|
I auto-fixed 5 issues. 2 need your input:
|
|
|
|
1. [CRITICAL] app/models/post.rb:42 — Race condition in status transition
|
|
Fix: Add `WHERE status = 'draft'` to the UPDATE
|
|
→ A) Fix B) Skip
|
|
|
|
2. [INFORMATIONAL] app/services/generator.rb:88 — LLM output not type-checked before DB write
|
|
Fix: Add JSON schema validation
|
|
→ A) Fix B) Skip
|
|
|
|
RECOMMENDATION: Fix both — #1 is a real race condition, #2 prevents silent data corruption.
|
|
```
|
|
|
|
If 3 or fewer ASK items, you may use individual AskUserQuestion calls instead of batching.
|
|
|
|
### Step 5d: Apply user-approved fixes
|
|
|
|
Apply fixes for items where the user chose "Fix." Output what was fixed.
|
|
|
|
If no ASK items exist (everything was AUTO-FIX), skip the question entirely.
|
|
|
|
### Verification of claims
|
|
|
|
Before producing the final review output:
|
|
- If you claim "this pattern is safe" → cite the specific line proving safety
|
|
- If you claim "this is handled elsewhere" → read and cite the handling code
|
|
- If you claim "tests cover this" → name the test file and method
|
|
- Never say "likely handled" or "probably tested" — verify or flag as unknown
|
|
|
|
**Rationalization prevention:** "This looks fine" is not a finding. Either cite evidence it IS fine, or flag it as unverified.
|
|
|
|
### Greptile comment resolution
|
|
|
|
After outputting your own findings, if Greptile comments were classified in Step 2.5:
|
|
|
|
**Include a Greptile summary in your output header:** `+ N Greptile comments (X valid, Y fixed, Z FP)`
|
|
|
|
Before replying to any comment, run the **Escalation Detection** algorithm from greptile-triage.md to determine whether to use Tier 1 (friendly) or Tier 2 (firm) reply templates.
|
|
|
|
1. **VALID & ACTIONABLE comments:** These are included in your findings — they follow the Fix-First flow (auto-fixed if mechanical, batched into ASK if not) (A: Fix it now, B: Acknowledge, C: False positive). If the user chooses A (fix), reply using the **Fix reply template** from greptile-triage.md (include inline diff + explanation). If the user chooses C (false positive), reply using the **False Positive reply template** (include evidence + suggested re-rank), save to both per-project and global greptile-history.
|
|
|
|
2. **FALSE POSITIVE comments:** Present each one via AskUserQuestion:
|
|
- Show the Greptile comment: file:line (or [top-level]) + body summary + permalink URL
|
|
- Explain concisely why it's a false positive
|
|
- Options:
|
|
- A) Reply to Greptile explaining why this is incorrect (recommended if clearly wrong)
|
|
- B) Fix it anyway (if low-effort and harmless)
|
|
- C) Ignore — don't reply, don't fix
|
|
|
|
If the user chooses A, reply using the **False Positive reply template** from greptile-triage.md (include evidence + suggested re-rank), save to both per-project and global greptile-history.
|
|
|
|
3. **VALID BUT ALREADY FIXED comments:** Reply using the **Already Fixed reply template** from greptile-triage.md — no AskUserQuestion needed:
|
|
- Include what was done and the fixing commit SHA
|
|
- Save to both per-project and global greptile-history
|
|
|
|
4. **SUPPRESSED comments:** Skip silently — these are known false positives from previous triage.
|
|
|
|
---
|
|
|
|
## Step 5.5: TODOS cross-reference
|
|
|
|
Read `TODOS.md` in the repository root (if it exists). Cross-reference the PR against open TODOs:
|
|
|
|
- **Does this PR close any open TODOs?** If yes, note which items in your output: "This PR addresses TODO: <title>"
|
|
- **Does this PR create work that should become a TODO?** If yes, flag it as an informational finding.
|
|
- **Are there related TODOs that provide context for this review?** If yes, reference them when discussing related findings.
|
|
|
|
If TODOS.md doesn't exist, skip this step silently.
|
|
|
|
---
|
|
|
|
## Step 5.6: Documentation staleness check
|
|
|
|
Cross-reference the diff against documentation files. For each `.md` file in the repo root (README.md, ARCHITECTURE.md, CONTRIBUTING.md, CLAUDE.md, etc.):
|
|
|
|
1. Check if code changes in the diff affect features, components, or workflows described in that doc file.
|
|
2. If the doc file was NOT updated in this branch but the code it describes WAS changed, flag it as an INFORMATIONAL finding:
|
|
"Documentation may be stale: [file] describes [feature/component] but code changed in this branch. Consider running `/document-release`."
|
|
|
|
This is informational only — never critical. The fix action is `/document-release`.
|
|
|
|
If no documentation files exist, skip this step silently.
|
|
|
|
---
|
|
|
|
{{ADVERSARIAL_STEP}}
|
|
|
|
## Important Rules
|
|
|
|
- **Read the FULL diff before commenting.** Do not flag issues already addressed in the diff.
|
|
- **Fix-first, not read-only.** AUTO-FIX items are applied directly. ASK items are only applied after user approval. Never commit, push, or create PRs — that's /ship's job.
|
|
- **Be terse.** One line problem, one line fix. No preamble.
|
|
- **Only flag real problems.** Skip anything that's fine.
|
|
- **Use Greptile reply templates from greptile-triage.md.** Every reply includes evidence. Never post vague replies.
|