Files
gstack/ship/sections/test-coverage.md.tmpl
T
Garry Tanandcapy-ai[bot] 5b9b8372c4 feat: honest work and gate integrity — gate-edit detector, review/ship gate pass, zero-run green, claims-need-evidence kernel (#3114)
* docs(designs): honest work and gate integrity plan (autoplan-reviewed)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(e2e): review-gate-integrity rule case registered in touchfiles and the PR profile

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* feat(honesty): claims-need-evidence kernel, no_claim card field, CEO value test, implementer dispatch rules, ACCEPTANCE_EDITED, Honest Work ethos

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* feat(gate-diff): gate-edit detector lib + CLI, patch fixtures, calibration script, tests_ran in gstack-evidence

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(gate-diff): RH-15? only on CI/config floors after calibration

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* feat(review,ship): Gate Integrity category, GATE_SCAN_BLOCK, one-way gate disposition, actor stamping, ZERO-RUN lanes, draft-while-open

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* chore: re-measure parity and context budgets on the integrated tree

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(touchfiles): gate-diff fixtures and helpers select review-gate-integrity

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: archaeology lint wording, paid-file census, harness-version non-behavioral record

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: pin review Step 3.5 rename, plan-path placeholder, mktemp template, per-stream evidence tail

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test: ACCEPTANCE_EDITED fence substitutes the <plan-path> placeholder

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(gate-diff): shallow boundary commits report no_base instead of a whole-tree inventory; recalibrate

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
2026-10-10 22:33:01 +00:00

119 lines
7.3 KiB
Cheetah

## Step 7: Test Coverage Audit
### Shared subagent dispatch
For Steps 7, 8 and 10, use the Agent tool with `run_in_background: false`.
Omitting the flag runs the subagent in the background. The explicit flag waits
for a result while keeping a fresh context. Do not invoke the target as a Skill
or run it inline instead. Inline work is allowed only under that section's
documented fallback, after a failed subagent has stopped.
Dispatch the audit through Agent with `subagent_type: "general-purpose"` and
`run_in_background: false`, using the shared foreground-dispatch rule above.
Wait for its LAST-line JSON before applying the coverage gate.
**Generation allowance:** Maximum 2 generation passes total per invocation.
Count each generation-authorized attempt before dispatch/inline execution, including
the initial audit, failures and zero-test results. Re-entry never resets it.
Two passes already used means no further generation; read-only reassessment uses no pass.
**Subagent prompt:** Supply `<base>`, Step 4's framework/bootstrap decision,
permitted paths/commands, remaining gaps, passes used and generation allowance,
plus the CLAUDE.md `## Test Coverage` values the gate below reads (`Generation cap:`,
`Base control:`, `Base control budget:`). No allowance means audit only; missing
permission is not approval. Preserve the 30-path/5-tests-per-pass/2-minute per-test caps.
**Before the first dispatch,** sweep base-control worktrees a previous interrupted run left behind:
```bash
git worktree prune
find "${TMPDIR:-/tmp}" -maxdepth 1 -name 'gstack-base-control.*' -mmin +10 2>/dev/null | while IFS= read -r d; do git worktree remove --force "$d/wt" >/dev/null 2>&1; rm -rf "$d"; done
```
````text
You are running a ship-workflow test coverage audit. Run `git diff origin/<base>` to include uncommitted tracked changes; also read relevant non-ignored untracked source/tests. Do not commit or push. Perform only this audit; return unresolved user decisions to the parent instead of asking or advancing to another workflow step.
Generation: <allowed|audit-only>; passes used: <N> of 2. Audit-only overrides every generation instruction below.
{{TEST_COVERAGE_AUDIT_SHIP}}
Rules for the implementing agent:
{{IMPLEMENTER_DISPATCH_RULES}}
Report any such change in the prose before the JSON line; the JSON fields below are unchanged.
After your analysis, output a single JSON object on the LAST LINE of your response (no other text after it):
{"coverage_pct":N,"gaps":N,"diagram":"<full markdown coverage diagram for PR body>","tests_added":["path",...],"coverage_pct_value":N,"weak_gaps":[{"path":"...","existing_test":"...","reason":"star_one|gate_failed|unrated"}],"tests_extended":["path",...],"tests_rejected":[{"path_or_gap":"...","reason_code":"...","reason":"..."}],"regression_proof":{"red_at_head":N,"base_green":N,"base_unavailable":N}}
`coverage_pct` is Y (paths with any test), `coverage_pct_value` is X (paths with a ★★/★★★ test), `gaps` counts only paths with no test. Use null for an undetermined or skipped coverage percentage, not zero. Include every remaining gap in the diagram so the parent can target a second pass.
````
**Parent processing:**
1. Read the subagent's final output. Parse the LAST line as JSON.
2. Store `coverage_pct`, `coverage_pct_value`, `gaps`, `weak_gaps`, `tests_added`,
`tests_extended`, `tests_rejected` and `regression_proof`. A missing new key counts
as empty; say so in the summary (an older installed prompt must not fail the gate).
A key with the wrong type (for example `weak_gaps` not an array) is ignored the same
way and printed as: {{TEST_VALUE_MESSAGE:malformedKey}}
3. **Machine checks** on every test written in this run (`tests_added` and
`tests_extended`): a value-card header with the four required fields non-empty plus
an optional `no_claim` (an older four-field card stays valid; else
`incomplete_card`); `protects` unique across the run after casefolding and stripping
punctuation and repeated whitespace (a later duplicate is `duplicate_protects`); seam
`none`, or a named seam with at least one non-test caller (N = 0 or an unavailable
caller check is `needs_seam`). Move each failure to `tests_rejected` with its
`reason_code`, then remove it before anything else reads the diff: an untracked new
file is deleted; for a tracked file, revert only this run's hunk with Edit, never
the whole file. Write one rejected test path per line, relative to the repository
root, into a private file:
{{FREE_TEXT_FILE:REJECTED_FILE=rejected-tests}}
Then run, substituting the printed name for `<rejected-file-name>`:
```bash
REJECTED_FILE="$(git rev-parse --show-toplevel 2>/dev/null || pwd)/.gstack/tmp/<rejected-file-name>"
[ -s "$REJECTED_FILE" ] || { echo "Nothing removed: $REJECTED_FILE is empty; write the paths, then rerun." >&2; exit 1; }
while IFS= read -r f; do
[ -n "$f" ] || continue
case "$f" in /*|..|../*|*/..|*/../*) echo "SKIPPED (outside repo): $f"; continue ;; esac
if git ls-files --error-unmatch -- "$f" >/dev/null 2>&1; then echo "REVERT_HUNK: $f"; else rm -f -- "$f" && echo "REMOVED: $f"; fi
done < "$REJECTED_FILE"
rm -f "$REJECTED_FILE"
```
No `tests_rejected` path may remain on disk as a new file. If every test written in
a pass is rejected, print {{TEST_VALUE_MESSAGE:allRejected}}
4. **Rating dispatch.** When this run wrote tests that survived the machine checks,
dispatch one read-only Agent (`subagent_type: "general-purpose"`,
`run_in_background: false`) with no generation permission; it uses no generation
pass. Give it the diagram and the surviving test paths. It rates each against the
★ rubric and the test value bar and returns a LAST-line JSON
`{"coverage_pct_value":N,"weak_gaps":[...]}` recomputed with its ratings; use those
two values. Until rated, this run's tests count as weak (`unrated`). If it fails,
times out or returns invalid JSON, the gate is skipped for this run ("rating
unavailable").
5. Embed `diagram` verbatim in the PR body's `## Test Coverage` section (Step 19).
6. Print a one-line summary: `Coverage: {X}% value-weighted ({Y}% including {W} weakly covered paths), {gaps} gaps. {tests_added.length} tests added.`
Bindings for the PR body's Test value line: K = `tests_added.length`,
R = `tests_rejected.length`, E = `tests_extended.length`, W = `weak_gaps.length`.
7. **Rescan gate edits after the child returns.** A child's green is a claim until
you have read its gate hunks. Run the scan over the whole candidate; never scan
only the paths the child reported:
{{GATE_SCAN_BLOCK}}
Read every read-level hunk with the checklist's Gate Integrity question. Carry the
`GATE_SUMMARY:` line and any hunk that fails it into Step 9's Gate Integrity
category; the disposition question is asked there, never here. A relaxation the
child made in a test it did not author is a Step 9 finding even when its tests pass.
**Audit failure:** On failure, invalid JSON or no completion after ~10 minutes,
stop the child and confirm it stopped before running the same audit inline.
Fallback recovers the audit; it does not pass or bypass the coverage gate.
Apply that gate to the recovered results, including its undetermined-percentage
and test-only rules. Preserve partial results as incomplete, not passing coverage.
{{TEST_COVERAGE_GATE_SHIP}}
---