mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-16 09:55:29 +02:00
Garry TanandOpenAI Codex
9f81911136
* feat: add a restricted and supervised Claude Code runner Preserve configured authentication and models while enforcing tool access, strict completion JSON, bounded output and process cleanup. Cover argv, failure handling, session metadata and Windows process containment. * feat: route outside reviews by harness and migrate wrapper installs Use Claude Code from Codex and Codex from other supported hosts, with shared invocation rendering, positive gate validation and per-phase provenance. Rename /claude to /claude-code, repair managed shared and copied installations safely, and generate native Kiro skills. Add installed-workflow, failure-injection and live cross-harness regression coverage. * test: recognize CEO mode labels without terminal spacing The paid workflow rendered SCOPEEXPANSION at option 4, but its driver required a literal space. Match the leading mode title without cursor-spacing artifacts and ignore adjacent preview text. Preserve missing-target failures and downstream posture assertions. * test: isolate plan-count fixtures before starting review workflows Seed the complete test plan in a private git repository before launching Claude, so a bare slash command cannot review the live workspace while a delayed fixture message remains queued. Preserve count thresholds, parsers and budgets. Add initial-context and installed-discovery tests, and retain startup/terminal diagnostics on failed evaluations. * test: stabilize review fixtures and Claude eval startup Preserve source boundaries in workflow judge inputs, isolate CEO mode plans, and wait for interactive trust input readiness. Keep startup failure evidence and retain existing models, budgets, and assertions. Co-Authored-By: OpenAI Codex <noreply@openai.com> * test: classify collapsed review modes and isolate seeded findings Keep review questions out of the setup count when terminal cursor positioning removes spaces. State existing webhook safeguards so the five-finding control measures its seeded defects without accidental extra security and concurrency gaps. Preserve question bands and the paired control. Co-Authored-By: OpenAI Codex <noreply@openai.com> * test: isolate browser daemon state across free shards Co-Authored-By: OpenAI Codex <noreply@openai.com> * test: stabilize native review counting and interactive navigation Co-Authored-By: OpenAI Codex <noreply@openai.com> * chore: prepare v1.82.0.0 release Co-Authored-By: OpenAI Codex <noreply@openai.com> * fix: eliminate browser and process-cleanup test flakes Pin every CI surface to Bun 1.4.0 to avoid extra-stdio finalizers closing reused live sockets. Add an isolated GC/listener regression that fails on Bun 1.3.13, and prevent coordinated rollback to an affected CI runtime. Check renderer cleanup against the render's own staging directory so concurrent renders cannot invalidate the assertion. Make the no-pgrep process-tree walk tolerate disappearing /proc entries, and synchronize its test fixture through child readiness and pipe EOF instead of sleeps. Validation: 9,157 passed, 31 skipped, zero failures across 556 files with retries disabled. Build, all-host generation freshness, and skill checks passed. All three races have failing-before/passing-after regressions. * fix: count completed native review questions in evals * fix: drive review navigation from confirmed native choices * fix: require complete section-loading eval reports * test: isolate telemetry HTTP transport from local assertions * fix: keep review input on the active native question * test: let tunnel revocation daemon choose an available port * test: allocate available ports for pairing and watchdog fixtures * fix: stabilize planning eval navigation and phase reporting * test: isolate installed runtime paths in planning evals * test: stabilize review evidence and concurrent refresh fixtures * fix: resolve design findings before editing the plan * fix: honor and persist disabled outside plan reviews * fix: preserve planning decisions and terminal evidence Load installed host reviews at autoplan phase entry and wait for completed reviewers and saved artifacts. Reuse approved remedies while preserving individual finding decisions. Drive interactive evals from the current terminal viewport, bind native questions across scrolling, and require complete native report evidence. Cover captured stale menus, permission lifecycles, setup classification, and disabled-review tool availability with deterministic regressions. Advance release metadata and the upgrade migration to the unclaimed 1.83.0.0 slot. * fix: drive native review questions and preserve current plans Use the native single-choice keyboard protocol and current terminal viewport, with per-question navigation inside packets and completed-call coverage. Keep permissions, multi-select menus, and Submit controls distinct. Send Autoplan reviewers the amended implementation plan, keep its review record separate, and supply retained application contracts in the chain fixture. Clarify individual DevEx decisions and complete CEO fix options; use one active plan destination for the section-loading report. * fix: preserve complete plan-review decisions * fix: recognize native plan dialogs and reviewer controls * fix: preserve review decisions and phase completion * fix: recognize completed reviews without losing findings * fix: preserve review continuity and native eval completion * test: fix native review completion and eval retry isolation * test: handle native review menus and complete eval fixtures * test: fix native review setup, completion, and isolation failures * test: limit native skill discovery to runtime assets * fix: bind Autoplan reviews to full ordered phase inputs * test: fix planning eval routing, counting, and timeout handling * chore: advance queued release to v1.84.0.0 * fix: preserve complete review inputs and planning decisions * fix: reconcile review approvals and preserve phase obligations * fix: preserve review obligations and unblock eval permissions Carry recorded Autoplan requirements into blind phase inputs, require Eng review approvals before exit, and exercise combined asynchronous flows in CEO reviews. Correct native finding and handoff classification and unblock repeated report edits using scoped request identities. * fix: retain plan requirements and complete native review dialogs * fix: complete native review prompts and retain plan references * fix: preserve review inputs and classify native eval evidence * fix: check competing completion orders in CEO reviews * fix: recognize review decisions and require phase methodology Require the current phase methodology before Autoplan snapshots. Correct substantive decision, closed handoff, and cache-finding classification, and honor the recommended implementation approach in native review dialogs. Add captured-transcript regressions without changing review thresholds, provider models, retries, or deadlines. * test: bind native review decisions and close completed handoffs * fix: complete review dialogs and verify methodology delivery * fix: preserve review evidence and unblock native eval prompts * fix: handle native review question completions * fix: recognize native review narration and controls * fix: count native review decisions and isolate eval fixtures * test: verify seeded review coverage and current artifact permissions * test: isolate model and brain-aware skill renders * fix: repair native workflow evaluation and clarify review steps * fix: stabilize workflow eval evidence and review guidance * test: repair native workflow observation and fixture isolation * fix: recognize completed workflow evidence and owned skill reads * test: repair seeded workflow delivery and completion evidence * test: recognize current review evidence across native forms * test: handle native review variants and permission redraws * fix: honor review preferences and recognize native eval evidence * test: recognize completed review decisions and queued permissions * test: match current review contracts and partial-line edits * test: recognize completed workflow evidence and bounded human waits * fix: preserve review entry gates and native eval interactions * fix: recognize native workflow evidence and preserve review gates * test: recognize current review evidence and preconfigure workflow fixtures * test: recognize completed review findings and scoped artifact permissions * fix: stabilize native workflow review and permission evidence * fix: recognize current review evidence and scoped edit confirmations Clarify Design and engineering review entry instructions and Design scoring. Recognize required legacy coverage and public Autoplan completion recaps. Bind the pending Edit confirmation to its exact file, ordered digest, and one-request approval when a preceding command display remains visible. Keep reviews within their existing size limits and preserve scope gates when extracting workflow fixtures from either supported preamble header. Keep failure outcomes, review thresholds, provider choices, and eval budgets. * fix: recover review workflow progress and eval evidence * fix: recognize valid review evidence and scope selection * test: fix review evidence parsing and repeated artifact prompts * test: recognize valid review decisions and pending native cards * fix(plan-eng-review): keep final navigation consistent with approved tasks * test: recognize valid review evidence and bind legacy diff requests * fix: stabilize review eval evidence and harness repair guidance * docs: update project documentation for v1.85.0.0 Co-Authored-By: OpenAI Codex <noreply@openai.com> * test: fix Windows CI fixtures and credential scan Rebase captured JSON values and filesystem evidence using the appropriate path convention. Compile native fake CLIs on Windows and synchronize pipe holder readiness, with cleanup retained when assertions fail. Assemble synthetic credential fixtures at runtime so the added-line scan keeps enforcing the same gate without flagging its own rejection controls. Discover generated skills directly for the empty-find regression check, avoiding a recursive scan through saved evaluation artifacts and dependencies. * fix: preserve source renders on Windows Compare canonical generator paths using native separators so an output sidecar pointing at the source cannot overwrite its skill or metadata. Keep the regression fixture isolated from the real checkout and expose freshness diagnostics before asserting subprocess status. Detach Windows drain-test pipe holders from the fake provider's automatic child cleanup while preserving the enclosing runner job and its assertions. * fix: clarify outside review fallback and CEO decisions Render one applicable own-harness fallback path and retain native review, disabled policy, and missing-coverage semantics. Align report field names and mode labels, and make the existing per-cut scope approval explicit. Regenerate skill outputs and keep the workflow judge's model, thresholds, and retry policy unchanged. * chore: move release to free version slot (v1.86.0.0) PR #2852 now claims v1.85.0.0. Align the release metadata and rename migration so upgrades from that version still receive it. Co-Authored-By: OpenAI Codex <noreply@openai.com> * fix: include engineering review prerequisites and restore branch context * fix: recognize coverage diagrams and clarify design review instructions * fix: preserve file identities and join Windows test processes --------- Co-authored-by: OpenAI Codex <noreply@openai.com>
1129 lines
142 KiB
JSON
1129 lines
142 KiB
JSON
{
|
|
"provenance": {
|
|
"collector": ".context/ship-source-af-delta-paid-20260909-v1/evals/job-72/shards/skill-e2e-workflow/1.85.0-edinburgh-v1-e2e-workflow-2026-09-10-0053.json",
|
|
"collectorSha256": "f83c96ef73a5a1ac159d56174a689f433c52729a99717743f8e67113b52c5db0",
|
|
"sourceCommit": "ca16058341c2e073051340d9653bbbc900f3ca36",
|
|
"projection": "Only public init/tool requests/results and final output. Private thinking not inspected.",
|
|
"history": "First two actual successful captures were recorded passed before failing the Read-only count assertion; third actual timeout remains failed."
|
|
},
|
|
"files": {
|
|
"source": "\nexport function processPayment(amount: number, currency: string) {\n if (amount <= 0) throw new Error('Invalid amount');\n if (currency !== 'USD' && currency !== 'EUR') throw new Error('Unsupported currency');\n return { status: 'success', amount, currency };\n}\n\nexport function refundPayment(paymentId: string, reason: string) {\n if (!paymentId) throw new Error('Payment ID required');\n if (!reason) throw new Error('Reason required');\n return { status: 'refunded', paymentId, reason };\n}\n",
|
|
"tests": "\nimport { describe, test, expect } from 'vitest';\nimport { processPayment } from '../src/billing';\n\ndescribe('processPayment', () => {\n test('processes valid payment', () => {\n const result = processPayment(100, 'USD');\n expect(result.status).toBe('success');\n });\n // GAP: no test for invalid amount\n // GAP: no test for unsupported currency\n // GAP: refundPayment not tested at all\n});\n"
|
|
},
|
|
"rows": [
|
|
{
|
|
"attempt": 1,
|
|
"recordedPassed": true,
|
|
"cwd": "/tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn",
|
|
"sessionId": "20dd6310-1b3c-4ff1-a914-f2ba87ad9bba",
|
|
"result": {
|
|
"exitReason": "success",
|
|
"browseErrors": [],
|
|
"output": "The branch has no diff against main, so I audited the two files named in the request directly. Tests are mocked as passing per instructions.\n\n```\nCODE PATHS USER FLOWS\n[+] src/billing.ts [+] Payment checkout\n \u251c\u2500\u2500 processPayment(amount, currency) \u251c\u2500\u2500 [GAP] [\u2192E2E] Successful charge end-to-end\n \u2502 \u251c\u2500\u2500 [\u2605\u2605 TESTED] Valid USD payment \u2192 success \u2014 billing.test.ts:6 \u2514\u2500\u2500 [GAP] [\u2192E2E] Rejected payment surfaced to user\n \u2502 \u251c\u2500\u2500 [GAP] amount <= 0 \u2192 throws 'Invalid amount'\n \u2502 \u251c\u2500\u2500 [GAP] currency not USD/EUR \u2192 throws 'Unsupported currency' [+] Refund flow\n \u2502 \u2514\u2500\u2500 [GAP] Valid EUR payment \u2192 success (false branch of currency guard) \u251c\u2500\u2500 [GAP] [\u2192E2E] Refund issued end-to-end\n \u2514\u2500\u2500 refundPayment(paymentId, reason) \u2514\u2500\u2500 [GAP] Refund rejected (missing id/reason) surfaced to user\n \u251c\u2500\u2500 [GAP] Valid id + reason \u2192 refunded\n \u251c\u2500\u2500 [GAP] empty paymentId \u2192 throws 'Payment ID required'\n \u2514\u2500\u2500 [GAP] empty reason \u2192 throws 'Reason required'\n\nLLM integration: none in diff\n\nCOVERAGE: 1/11 paths tested (9%) | Code paths: 1/7 (14%) | User flows: 0/4 (0%)\nQUALITY: \u2605\u2605\u2605:0 \u2605\u2605:1 \u2605:0 | GAPS: 10 (3 E2E, 0 eval)\n```\n\nLegend: \u2605\u2605\u2605 behavior + edge + error | \u2605\u2605 happy path | \u2605 smoke check\n[\u2192E2E] = needs integration test | [\u2192EVAL] = needs LLM eval\n\n**Summary**\n\n- **The single existing test** covers only the USD happy path and asserts only the status field. It does not check that amount and currency echo back correctly.\n- **Both guard clauses in processPayment are untested.** Invalid amount and unsupported currency throw paths have no coverage, and the EUR branch of the currency check is never exercised.\n- **refundPayment has zero coverage.** All three of its paths, the happy path and both validation errors, are gaps.\n- **No regressions detected.** With an empty diff against main there is no modified pre-existing behavior to regress.\n- **Coverage gate would fail.** At 9% overall the result is well below the default 60% minimum, which in a full ship run would prompt the generate-tests or override decision.\n\nTest files before audit: 1. No tests were generated and no test plan artifact was written, per the request to produce the diagram only.\n\nCoverage: 9%, 10 gaps. 0 tests added.",
|
|
"transcript": [
|
|
{
|
|
"type": "system",
|
|
"subtype": "init",
|
|
"session_id": "20dd6310-1b3c-4ff1-a914-f2ba87ad9bba",
|
|
"cwd": "/tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn"
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "20dd6310-1b3c-4ff1-a914-f2ba87ad9bba",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"id": "msg_011CetqcQqNyiEnmy5J2wYvX",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_01NeaECsZKkMhoU6xN7DYHqb",
|
|
"name": "Bash",
|
|
"input": {
|
|
"command": "cat /tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn/ship/SKILL.md 2>/dev/null || find / -path '*/ship/SKILL.md' 2>/dev/null | head",
|
|
"description": "Read ship SKILL.md"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "20dd6310-1b3c-4ff1-a914-f2ba87ad9bba",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_01NeaECsZKkMhoU6xN7DYHqb",
|
|
"is_error": false,
|
|
"content": "<persisted-output>\nOutput too large (76.1KB). Full output saved to: /tmp/gstack-paid-shard-6fJnEY/tmp/gstack-hermetic-3647161-TszbDZ/.claude/projects/-tmp-gstack-paid-shard-6fJnEY-tmp-skill-e2e-coverage-wQ70Nn/20dd6310-1b3c-4ff1-a914-f2ba87ad9bba/tool-results/bs0sgawur.txt\n\nPreview (first 2KB):\n---\nname: ship\npreamble-tier: 4\nversion: 1.0.0\ndescription: \"Ship workflow: detect + merge base branch, run tests, review diff, bump VERSION, update CHANGELOG, commit, push, create PR. (gstack)\"\nallowed-tools:\n - Bash\n - Read\n - Write\n - Edit\n - Grep\n - Glob\n - Agent\n - AskUserQuestion\n - WebSearch\ntriggers:\n - ship it\n - create a pr\n - push to main\n - deploy this\n---\n<!-- AUTO-GENERATED from SKILL.md.tmpl \u2014 do not edit directly -->\n<!-- Regenerate: bun run gen:skill-docs -->\n\n\n## When to invoke this skill\n\nUse when asked to \"ship\", \"deploy\",\n\"push to main\", \"create a PR\", \"merge and push\", or \"get it deployed\".\nProactively invoke this skill (do NOT push/PR directly) when the user says code\nis ready, asks about deploying, wants to push code up, or asks to create a PR.\n\n## Preamble (run first)\n\n```bash\n_SS=\"$HOME/.claude/skills/gstack/bin/gstack-skill-start\"\n[ -x \"$_SS\" ] || _SS=\".claude/skills/gstack/bin/gstack-skill-start\"\n\"$_SS\" --skill \"ship\" --model \"claude\" --parent-pid \"$PPID\" \\\n || echo \"SKILL_START: unavailable \u2014 stale install; run ./setup or /gstack-upgrade (preamble degraded, continue the user's task)\"\n```\n\nRead the echoed `KEY: value` STATUS lines \u2014 they drive every preamble rule\nbelow. **Degraded mode:** if `SKILL_START_PROTO: 1` is missing from the output\n(script absent, stale install, or a different protocol number), apply safe\ndefaults: treat `SESSION_KIND` as `interactive`, do NOT assume Conductor,\nskip onboarding/telemetry steps (their gates are marker-based, so consent and\nonboarding prompts are DEFERRED to the next healthy run \u2014 never lost), tell\nthe user to run `./setup` or `/gstack-upgrade`, and proceed with their task.\nNote `SESSION_ID` and `TEL_START` from the output \u2014 the Telemetry step needs\nthem at skill end.\n\n**Instruction blocks:** the output may contain\n`GSTACK_INSTRUCTION_BEGIN: <id> <session-id>` \u2026 `GSTACK_INSTRUCTION_END`\nblocks \u2014 one-time onboarding and consent directives whose runtime gates fired.\n...\n</persisted-output>"
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "20dd6310-1b3c-4ff1-a914-f2ba87ad9bba",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"id": "msg_011CetqcuQLntqebFWUYEjLh",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_01UuYHohXEt4vWkJJ2VK4kKp",
|
|
"name": "Bash",
|
|
"input": {
|
|
"command": "grep -n \"3.4\\|Coverage Audit\" /tmp/gstack-paid-shard-6fJnEY/tmp/gstack-hermetic-3647161-TszbDZ/.claude/projects/-tmp-gstack-paid-shard-6fJnEY-tmp-skill-e2e-coverage-wQ70Nn/20dd6310-1b3c-4ff1-a914-f2ba87ad9bba/tool-results/bs0sgawur.txt | head -20",
|
|
"description": "Locate Step 3.4 in SKILL.md"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "20dd6310-1b3c-4ff1-a914-f2ba87ad9bba",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_01UuYHohXEt4vWkJJ2VK4kKp",
|
|
"is_error": false,
|
|
"content": "(Bash completed with no output)"
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "20dd6310-1b3c-4ff1-a914-f2ba87ad9bba",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"id": "msg_011CetqcuQLntqebFWUYEjLh",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_01NJEycXhvfSSwFdbuXkAncV",
|
|
"name": "Bash",
|
|
"input": {
|
|
"command": "cat -n /tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn/src/billing.ts",
|
|
"description": "Read billing source"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "20dd6310-1b3c-4ff1-a914-f2ba87ad9bba",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_01NJEycXhvfSSwFdbuXkAncV",
|
|
"is_error": false,
|
|
"content": " 1\t\n 2\texport function processPayment(amount: number, currency: string) {\n 3\t if (amount <= 0) throw new Error('Invalid amount');\n 4\t if (currency !== 'USD' && currency !== 'EUR') throw new Error('Unsupported currency');\n 5\t return { status: 'success', amount, currency };\n 6\t}\n 7\t\n 8\texport function refundPayment(paymentId: string, reason: string) {\n 9\t if (!paymentId) throw new Error('Payment ID required');\n 10\t if (!reason) throw new Error('Reason required');\n 11\t return { status: 'refunded', paymentId, reason };\n 12\t}"
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "20dd6310-1b3c-4ff1-a914-f2ba87ad9bba",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"id": "msg_011CetqcuQLntqebFWUYEjLh",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_01GkDhZtJUViZtuGAvfViD2d",
|
|
"name": "Bash",
|
|
"input": {
|
|
"command": "cat -n /tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn/test/billing.test.ts",
|
|
"description": "Read billing tests"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "20dd6310-1b3c-4ff1-a914-f2ba87ad9bba",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_01GkDhZtJUViZtuGAvfViD2d",
|
|
"is_error": false,
|
|
"content": " 1\t\n 2\timport { describe, test, expect } from 'vitest';\n 3\timport { processPayment } from '../src/billing';\n 4\t\n 5\tdescribe('processPayment', () => {\n 6\t test('processes valid payment', () => {\n 7\t const result = processPayment(100, 'USD');\n 8\t expect(result.status).toBe('success');\n 9\t });\n 10\t // GAP: no test for invalid amount\n 11\t // GAP: no test for unsupported currency\n 12\t // GAP: refundPayment not tested at all\n 13\t});"
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "20dd6310-1b3c-4ff1-a914-f2ba87ad9bba",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"id": "msg_011CetqdcEFaxFqob5kA3NEL",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_0112c9HbajQKJtTnvyYYfdug",
|
|
"name": "Bash",
|
|
"input": {
|
|
"command": "grep -n -i \"coverage\\|^### Step\\|^## Step\" /tmp/gstack-paid-shard-6fJnEY/tmp/gstack-hermetic-3647161-TszbDZ/.claude/projects/-tmp-gstack-paid-shard-6fJnEY-tmp-skill-e2e-coverage-wQ70Nn/20dd6310-1b3c-4ff1-a914-f2ba87ad9bba/tool-results/bs0sgawur.txt | head -40",
|
|
"description": "Find coverage audit section in SKILL.md"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "20dd6310-1b3c-4ff1-a914-f2ba87ad9bba",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_0112c9HbajQKJtTnvyYYfdug",
|
|
"is_error": false,
|
|
"content": "118:Completeness: A=X/10, B=Y/10 (or: Note: options differ in kind, not coverage \u2014 no completeness score)\n133:Completeness: use `Completeness: N/10` only when options differ in coverage. 10 = complete, 7 = happy path, 3 = shortcut. If options differ in kind, write: `Note: options differ in kind, not coverage \u2014 no completeness score.`\n174:- [ ] Completeness scored (coverage) OR kind-note present (kind)\n285:AI makes completeness cheap, so the complete thing is the goal. Recommend full coverage (tests, edge cases, error paths) \u2014 boil the ocean one lake at a time. The only thing out of scope is genuinely unrelated work (rewrites, multi-quarter migrations); flag that as separate scope, never as an excuse for a shortcut.\n287:When options differ in coverage, include `Completeness: X/10` (10 = all edge cases, 7 = happy path, 3 = shortcut). When options differ in kind, write: `Note: options differ in kind, not coverage \u2014 no completeness score.` Do not fabricate scores.\n459:## Step 0: Detect platform and base branch\n511:- AI-assessed coverage below target (see Step 7 for minimum/target decisions)\n525:- Test coverage gaps within target threshold (auto-generate and commit, or flag in PR body)\n529:(tests, coverage audit, plan completion, pre-landing review, adversarial review,\n548:| auditing test coverage of the diff (Step 7) | `sections/test-coverage.md` |\n558:## Step 0.9: Apple target detection\n570:## Step 1: Pre-flight\n588:Render each record using its recorded host, source, outside_provider, outside_status, and phase. Historical source \"claude\" means a native Claude subagent; source \"claude-code\" means the external CLI. Never infer a historical provider from the current harness. Unknown model identity remains unknown. Missing/disabled/skipped outside coverage is distinct from native completion.\n594:Read `autoplan-voices` and `design-outside-voices` for the coverage detail below the dashboard. Group by workflow run and phase, not merely skill. Show each phase\u2019s recorded provider and outside_status; partial coverage must remain partial. These records do not change the engineering gate.\n619:- **Outside Voice (default-on):** Independent plan review through the host-selected provider after /plan-ceo-review and /plan-eng-review. The codex_reviews switch disables the entire extra step. Provider failure uses the existing native fallback and reports missing outside coverage. Never gates shipping.\n648:## Step 2: Distribution Pipeline Check\n676:## Step 3: Merge the base branch (BEFORE tests)\n693:> **STOP.** Before auditing test coverage of the diff (Step 7), Read `~/.claude/skills/gstack/ship/sections/test-coverage.md` and execute it\n708:## Step 12: Version bump (auto-decide)\n752:## Step 14: TODOS.md (auto-update)\n802:## Step 15: Commit (bisectable chunks)\n804:### Step 15.0: WIP Commit Squash (continuous checkpoint mode only)\n881:### Step 15.1: Bisectable Commits\n919:## Step 16: Verification Gate\n962:## Step 17: Push\n1049:## Step 20: Persist ship metrics\n1051:Log coverage and plan completion data so `/retro` can track trends.\n1061:~/.claude/skills/gstack/bin/gstack-review-log '{\"skill\":\"ship\",\"timestamp\":\"'\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\"'\",\"coverage_pct\":COVERAGE_PCT,\"plan_items_total\":PLAN_TOTAL,\"plan_items_done\":PLAN_DONE,\"verification_result\":\"VERIFY_RESULT\",\"version\":\"VERSION\",\"branch\":\"'\"$(git rev-parse --abbrev-ref HEAD)\"'\"}'\n1065:- **COVERAGE_PCT**: coverage percentage from Step 7 diagram (integer, or -1 if undetermined)\n1078:## Step 21: Plan-tune discoverability nudge (first-successful-ship only)\n1123:- **Step 7 generates coverage tests.** They must pass before committing. Never commit failing tests."
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "20dd6310-1b3c-4ff1-a914-f2ba87ad9bba",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"id": "msg_011CetqeE6GVwiHsDjGn95FE",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_01XeGXRpQuewHbFXyfoib3wq",
|
|
"name": "Bash",
|
|
"input": {
|
|
"command": "cat /tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn/ship/sections/test-coverage.md 2>/dev/null || cat ~/.claude/skills/gstack/ship/sections/test-coverage.md 2>/dev/null || find / -name test-coverage.md -path '*ship*' 2>/dev/null | head -3",
|
|
"description": "Read the test-coverage section instructions"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "20dd6310-1b3c-4ff1-a914-f2ba87ad9bba",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_01XeGXRpQuewHbFXyfoib3wq",
|
|
"is_error": false,
|
|
"content": "<!-- AUTO-GENERATED from test-coverage.md.tmpl \u2014 do not edit directly -->\n<!-- Regenerate: bun run gen:skill-docs -->\n## Step 7: Test Coverage Audit\n\n**Dispatch this step as a subagent** using the Agent tool with `subagent_type: \"general-purpose\"`. The subagent runs the coverage audit in a fresh context window \u2014 the parent only sees the conclusion, not intermediate file reads. This is context-rot defense.\n\n**Foreground required:** pass `run_in_background: false` on the Agent call \u2014 subagents run in the BACKGROUND by default since Claude Code v2.1.198. (Merely omitting the flag no longer produces a foreground run; it must be explicitly false.) The dispatch happens ONLY via the Agent tool: invoking the target as a Skill, or executing its workflow inline in your own context, is WRONG even though the skill may appear in your available-skills list \u2014 inline execution forfeits the fresh-context isolation this dispatch exists for, and the explicit flag already makes the Agent call block. (Where a step defines an inline FALLBACK, it applies only after a dispatched subagent has failed.) The parent needs this audit's LAST-line JSON before continuing.\n\n**Subagent prompt:** Pass the following instructions to the subagent, with `<base>` substituted with the base branch:\n\n````text\nYou are running a ship-workflow test coverage audit. Run `git diff <base>...HEAD` as needed. Do not commit or push. Perform only this audit; return unresolved user decisions to the parent instead of asking or advancing to another workflow step.\n\n100% coverage is the goal \u2014 every untested path is a path where bugs hide and vibe coding becomes yolo coding. Evaluate what was ACTUALLY coded (from the diff), not what was planned.\n\n### Test Framework Detection\n\nBefore analyzing coverage, detect the project's test framework:\n\n1. **Read CLAUDE.md** \u2014 look for a `## Testing` section with test command and framework name. If found, use that as the authoritative source.\n2. **If CLAUDE.md has no testing section, auto-detect:**\n\n```bash\nsetopt +o nomatch 2>/dev/null || true # zsh compat\n# Detect project runtime (markers are evidence, not commands to run blind)\n[ -f manage.py ] && echo \"RUNTIME:python FRAMEWORK:django\"\n{ [ -f pyproject.toml ] || [ -f pytest.ini ] || [ -f tox.ini ] || [ -f setup.cfg ] || [ -f requirements.txt ]; } && echo \"RUNTIME:python\"\n[ -f Gemfile ] || [ -f Rakefile ] || [ -f .rspec ] && echo \"RUNTIME:ruby\"\n[ -f package.json ] && echo \"RUNTIME:node\"\n[ -f go.mod ] && echo \"RUNTIME:go\"\n[ -f Cargo.toml ] && echo \"RUNTIME:rust\"\n[ -f pom.xml ] && echo \"RUNTIME:jvm BUILD:maven\"\n{ [ -f build.gradle ] || [ -f build.gradle.kts ]; } && echo \"RUNTIME:jvm BUILD:gradle\"\n# Check for existing test infrastructure \u2014 config files, scripts, AND test files\nls jest.config.* vitest.config.* playwright.config.* cypress.config.* .rspec pytest.ini tox.ini phpunit.xml 2>/dev/null\n[ -f package.json ] && grep -q '\"test\"[[:space:]]*:' package.json && echo \"SCRIPT:package.json test\"\n[ -f Makefile ] && grep -qE '^(test|check):' Makefile && echo \"TARGET:make test\"\ngit ls-files | grep -cE '(^|/)(tests?|spec|__tests__)/|(^|/)tests?\\.py$|(^|/)test_[^/]+\\.py$|_test\\.(go|py|rb|ts|js|exs)$|\\.(test|spec)\\.[jt]sx?$|_spec\\.rb$|Test\\.(java|kt)$' | sed 's/^/TESTFILES:/'\n```\n\n3. **If no framework detected:** use the bootstrap decision already made in Step 4; report diagram-only coverage if setup was declined. Do not restart bootstrap from this audit.\n\n**0. Before/after test count:**\n\n```bash\n# Count test files before any generation\ngit ls-files 2>/dev/null | grep -E '(\\.test\\.|\\.spec\\.|_test\\.|_spec\\.)' | wc -l\n```\n\nStore this number for the PR body.\n\n**1. Trace every codepath changed** using `git diff origin/<base>...HEAD`:\n\nRead every changed file. For each one, trace how data flows through the code \u2014 don't just list functions, actually follow the execution:\n\n1. **Read the diff.** For each changed file, read the full file (not just the diff hunk) to understand context.\n2. **Trace data flow.** Starting from each entry point (route handler, exported function, event listener, component render), follow the data through every branch:\n - Where does input come from? (request params, props, database, API call)\n - What transforms it? (validation, mapping, computation)\n - Where does it go? (database write, API response, rendered output, side effect)\n - What can go wrong at each step? (null/undefined, invalid input, network failure, empty collection)\n3. **Diagram the execution.** For each changed file, draw an ASCII diagram showing:\n - Every function/method that was added or modified\n - Every conditional branch (if/else, switch, ternary, guard clause, early return)\n - Every error path (try/catch, rescue, error boundary, fallback)\n - Every call to another function (trace into it \u2014 does IT have untested branches?)\n - Every edge: what happens with null input? Empty array? Invalid type?\n\nThis is the critical step \u2014 you're building a map of every line of code that can execute differently based on input. Every branch in this diagram needs a test.\n\n**2. Map user flows, interactions, and error states:**\n\nCode coverage isn't enough \u2014 you need to cover how real users interact with the changed code. For each changed feature, think through:\n\n- **User flows:** What sequence of actions does a user take that touches this code? Map the full journey (e.g., \"user clicks 'Pay' \u2192 form validates \u2192 API call \u2192 success/failure screen\"). Each step in the journey needs a test.\n- **Interaction edge cases:** What happens when the user does something unexpected?\n - Double-click/rapid resubmit\n - Navigate away mid-operation (back button, close tab, click another link)\n - Submit with stale data (page sat open for 30 minutes, session expired)\n - Slow connection (API takes 10 seconds \u2014 what does the user see?)\n - Concurrent actions (two tabs, same form)\n- **Error states the user can see:** For every error the code handles, what does the user actually experience?\n - Is there a clear error message or a silent failure?\n - Can the user recover (retry, go back, fix input) or are they stuck?\n - What happens with no network? With a 500 from the API? With invalid data from the server?\n- **Empty/zero/boundary states:** What does the UI show with zero results? With 10,000 results? With a single character input? With maximum-length input?\n\nAdd these to your diagram alongside the code branches. A user flow with no test is just as much a gap as an untested if/else.\n\n**3. Check each branch against existing tests:**\n\nGo through your diagram branch by branch \u2014 both code paths AND user flows. For each one, search for a test that exercises it:\n- Function `processPayment()` \u2192 look for `billing.test.ts`, `billing.spec.ts`, `test/billing_test.rb`\n- An if/else \u2192 look for tests covering BOTH the true AND false path\n- An error handler \u2192 look for a test that triggers that specific error condition\n- A call to `helperFn()` that has its own branches \u2192 those branches need tests too\n- A user flow \u2192 look for an integration or E2E test that walks through the journey\n- An interaction edge case \u2192 look for a test that simulates the unexpected action\n\nQuality scoring rubric:\n- \u2605\u2605\u2605 Tests behavior with edge cases AND error paths\n- \u2605\u2605 Tests correct behavior, happy path only\n- \u2605 Smoke test / existence check / trivial assertion (e.g., \"it renders\", \"it doesn't throw\")\n\n### E2E Test Decision Matrix\n\nWhen checking each branch, also determine whether a unit test or E2E/integration test is the right tool:\n\n**RECOMMEND E2E (mark as [\u2192E2E] in the diagram):**\n- Common user flow spanning 3+ components/services (e.g., signup \u2192 verify email \u2192 first login)\n- Integration point where mocking hides real failures (e.g., API \u2192 queue \u2192 worker \u2192 DB)\n- Auth/payment/data-destruction flows \u2014 too important to trust unit tests alone\n\n**RECOMMEND EVAL (mark as [\u2192EVAL] in the diagram):**\n- Critical LLM call that needs a quality eval (e.g., prompt change \u2192 test output still meets quality bar)\n- Changes to prompt templates, system instructions, or tool definitions\n\n**STICK WITH UNIT TESTS:**\n- Pure function with clear inputs/outputs\n- Internal helper with no side effects\n- Edge case of a single function (null input, empty array)\n- Obscure/rare flow that isn't customer-facing\n\n### REGRESSION RULE (mandatory)\n\n**IRON RULE:** When the coverage audit identifies a REGRESSION \u2014 code that previously worked but the diff broke \u2014 a regression test is written immediately. No AskUserQuestion. No skipping. Regressions are the highest-priority test because they prove something broke.\n\nA regression is when:\n- The diff modifies existing behavior (not new code)\n- The existing test suite (if any) doesn't cover the changed path\n- The change introduces a new failure mode for existing callers\n\nWhen uncertain whether a change is a regression, err on the side of writing the test.\n\n**4. Output ASCII coverage diagram:**\n\nInclude BOTH code paths and user flows in the same diagram. Mark E2E-worthy and eval-worthy paths:\n\n```\nCODE PATHS USER FLOWS\n[+] src/services/billing.ts [+] Payment checkout\n \u251c\u2500\u2500 processPayment() \u251c\u2500\u2500 [\u2605\u2605\u2605 TESTED] Complete purchase \u2014 checkout.e2e.ts:15\n \u2502 \u251c\u2500\u2500 [\u2605\u2605\u2605 TESTED] happy + declined + timeout \u251c\u2500\u2500 [GAP] [\u2192E2E] Double-click submit\n \u2502 \u251c\u2500\u2500 [GAP] Network timeout \u2514\u2500\u2500 [GAP] Navigate away mid-payment\n \u2502 \u2514\u2500\u2500 [GAP] Invalid currency\n \u2514\u2500\u2500 refundPayment() [+] Error states\n \u251c\u2500\u2500 [\u2605\u2605 TESTED] Full refund \u2014 :89 \u251c\u2500\u2500 [\u2605\u2605 TESTED] Card declined message\n \u2514\u2500\u2500 [\u2605 TESTED] Partial (non-throw only) \u2014 :101 \u2514\u2500\u2500 [GAP] Network timeout UX\n\nLLM integration: [GAP] [\u2192EVAL] Prompt template change \u2014 needs eval test\n\nCOVERAGE: 5/13 paths tested (38%) | Code paths: 3/5 (60%) | User flows: 2/8 (25%)\nQUALITY: \u2605\u2605\u2605:2 \u2605\u2605:2 \u2605:1 | GAPS: 8 (2 E2E, 1 eval)\n```\n\nLegend: \u2605\u2605\u2605 behavior + edge + error | \u2605\u2605 happy path | \u2605 smoke check\n[\u2192E2E] = needs integration test | [\u2192EVAL] = needs LLM eval\n\n**Fast path:** All paths covered \u2192 \"Step 7: All new code paths have test coverage \u2713\" Continue.\n\n**5. Generate tests for uncovered paths:**\n\nIf test framework detected (or bootstrapped in Step 4):\n- Prioritize error handlers and edge cases first (happy paths are more likely already tested)\n- Read 2-3 existing test files to match conventions exactly\n- Generate unit tests. Mock all external dependencies (DB, API, Redis).\n- For paths marked [\u2192E2E]: generate integration/E2E tests using the project's E2E framework (Playwright, Cypress, Capybara, etc.)\n- For paths marked [\u2192EVAL]: generate eval tests using the project's eval framework, or flag for manual eval if none exists\n- Write tests that exercise the specific uncovered path with real assertions\n- Run each test. Passes \u2192 keep the change and report its path; the parent commits in Step 15.\n- Fails \u2192 fix once. Still fails \u2192 revert, note gap in diagram.\n\nCaps: 30 code paths max, 20 tests generated max (code + user flow combined), 2-min per-test exploration cap.\n\nIf no test framework AND user declined bootstrap \u2192 diagram only, no generation. Note: \"Test generation skipped \u2014 no test framework configured.\"\n\n**Diff is test-only changes:** Return a skipped audit with null coverage, zero gaps, and \"No new application code paths to audit.\"\n\n**6. After-count and coverage summary:**\n\n```bash\n# Count test files after generation\ngit ls-files 2>/dev/null | grep -E '(\\.test\\.|\\.spec\\.|_test\\.|_spec\\.)' | wc -l\n```\n\nFor PR body: `Tests: {before} \u2192 {after} (+{delta} new)`\nCoverage line: `Test Coverage Audit: N new code paths. M covered (X%). K tests generated, awaiting parent commit.`\n\n### Test Plan Artifact\n\nAfter producing the coverage diagram, write a test plan artifact so `/qa` and `/qa-only` can consume it:\n\n```bash\neval \"$(~/.claude/skills/gstack/bin/gstack-slug 2>/dev/null)\" && mkdir -p ~/.gstack/projects/$SLUG\nUSER=$(whoami)\nDATETIME=$(date +%Y%m%d-%H%M%S)\n```\n\nWrite to `~/.gstack/projects/{slug}/{user}-{branch}-ship-test-plan-{datetime}.md`:\n\n```markdown\n# Test Plan\nGenerated by /ship on {date}\nBranch: {branch}\nRepo: {owner/repo}\n\n## Affected Pages/Routes\n- {URL path} \u2014 {what to test and why}\n\n## Key Interactions to Verify\n- {interaction description} on {page}\n\n## Edge Cases\n- {edge case} on {page}\n\n## Critical Paths\n- {end-to-end flow that must work}\n```\n\nAfter your analysis, output a single JSON object on the LAST LINE of your response (no other text after it):\n{\"coverage_pct\":N,\"gaps\":N,\"diagram\":\"<full markdown coverage diagram for PR body>\",\"tests_added\":[\"path\",...]}\nUse null for an undetermined or skipped coverage percentage, not zero. Include every remaining gap in the diagram so the parent can target a second pass.\n````\n\n**Parent processing:**\n\n1. Read the subagent's final output. Parse the LAST line as JSON.\n2. Store `coverage_pct` (for Step 20 metrics), `gaps` (user summary), `tests_added` (for the commit).\n3. Embed `diagram` verbatim in the PR body's `## Test Coverage` section (Step 19).\n4. Print a one-line summary: `Coverage: {coverage_pct}%, {gaps} gaps. {tests_added.length} tests added.`\n\n**If the subagent fails, times out, returns invalid JSON, or never completes (backgrounded despite the flag, or no final output after ~10 minutes \u2014 stop waiting; if a backgrounded task is still running, stop it first so a late result never races the fallback):** Fall back to running the audit inline in the parent. Do not block /ship on subagent failure \u2014 partial results are better than none.\n\n\n**7. Coverage gate:**\n\nThe parent owns this gate after receiving the audit result, including after an inline fallback. Generated tests stay uncommitted until Step 15. Any further generation uses the same audit prompt with the remaining gaps and pass count supplied.\n\nBefore proceeding, check CLAUDE.md for a `## Test Coverage` section with `Minimum:` and `Target:` fields. If found, use those percentages. Otherwise use defaults: Minimum = 60%, Target = 80%.\n\nUsing the coverage percentage from the diagram in substep 4 (the `COVERAGE: X/Y (Z%)` line):\n\n- **>= target:** Pass. \"Coverage gate: PASS ({X}%).\" Continue.\n- **>= minimum, < target:** Use AskUserQuestion:\n - \"AI-assessed coverage is {X}%. {N} code paths are untested. Target is {target}%.\"\n - RECOMMENDATION: Choose A because untested code paths are where production bugs hide.\n - Options:\n A) Generate more tests for remaining gaps (recommended)\n B) Ship anyway \u2014 I accept the coverage risk\n C) These paths don't need tests \u2014 mark as intentionally uncovered\n - If A: Dispatch one more generation pass targeting remaining gaps, then re-evaluate the result here. Maximum 2 generation passes total. At the cap, offer only B/C or stop; do not offer another generation pass.\n - If B: Continue. Include in PR body: \"Coverage gate: {X}% \u2014 user accepted risk.\"\n - If C: Continue. Include in PR body: \"Coverage gate: {X}% \u2014 {N} paths intentionally uncovered.\"\n\n- **< minimum:** Use AskUserQuestion:\n - \"AI-assessed coverage is critically low ({X}%). {N} of {M} code paths have no tests. Minimum threshold is {minimum}%.\"\n - RECOMMENDATION: Choose A because less than {minimum}% means more code is untested than tested.\n - Options:\n A) Generate tests for remaining gaps (recommended)\n B) Override \u2014 ship with low coverage (I understand the risk)\n - If A: Dispatch one more generation pass. Maximum 2 passes total. At the cap, offer only B or stop; do not offer another generation pass.\n - If B: Continue. Include in PR body: \"Coverage gate: OVERRIDDEN at {X}%.\"\n\n**Coverage percentage undetermined:** If the coverage diagram doesn't produce a clear numeric percentage (ambiguous output, parse error), **skip the gate** with: \"Coverage gate: could not determine percentage \u2014 skipping.\" Do not default to 0% or block.\n\n**Test-only diffs:** Skip the gate (same as the existing fast-path).\n\n**100% coverage:** \"Coverage gate: PASS (100%).\" Continue.\n\n---"
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "20dd6310-1b3c-4ff1-a914-f2ba87ad9bba",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"id": "msg_011CetqeewzSBYHCkFBqFuL6",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_017DuGUo6oEtV3aP1o1wu2KA",
|
|
"name": "Bash",
|
|
"input": {
|
|
"command": "git diff main...HEAD --stat; echo \"---\"; git log --oneline main..HEAD; echo \"--- test files:\"; git ls-files | grep -E '(\\.test\\.|\\.spec\\.|_test\\.|_spec\\.)' | wc -l; echo \"--- CLAUDE.md:\"; cat CLAUDE.md 2>/dev/null | head -20; ls",
|
|
"description": "Check diff vs main, test file count, CLAUDE.md"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "20dd6310-1b3c-4ff1-a914-f2ba87ad9bba",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_017DuGUo6oEtV3aP1o1wu2KA",
|
|
"is_error": false,
|
|
"content": "---\n--- test files:\n1\n--- CLAUDE.md:\nCHANGELOG.md\nVERSION\npackage.json\nreview\nship\nsrc\ntest\nvitest.config.ts"
|
|
}
|
|
]
|
|
}
|
|
}
|
|
],
|
|
"toolCalls": [
|
|
{
|
|
"tool": "Bash",
|
|
"input": {
|
|
"command": "cat /tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn/ship/SKILL.md 2>/dev/null || find / -path '*/ship/SKILL.md' 2>/dev/null | head",
|
|
"description": "Read ship SKILL.md"
|
|
}
|
|
},
|
|
{
|
|
"tool": "Bash",
|
|
"input": {
|
|
"command": "grep -n \"3.4\\|Coverage Audit\" /tmp/gstack-paid-shard-6fJnEY/tmp/gstack-hermetic-3647161-TszbDZ/.claude/projects/-tmp-gstack-paid-shard-6fJnEY-tmp-skill-e2e-coverage-wQ70Nn/20dd6310-1b3c-4ff1-a914-f2ba87ad9bba/tool-results/bs0sgawur.txt | head -20",
|
|
"description": "Locate Step 3.4 in SKILL.md"
|
|
}
|
|
},
|
|
{
|
|
"tool": "Bash",
|
|
"input": {
|
|
"command": "cat -n /tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn/src/billing.ts",
|
|
"description": "Read billing source"
|
|
}
|
|
},
|
|
{
|
|
"tool": "Bash",
|
|
"input": {
|
|
"command": "cat -n /tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn/test/billing.test.ts",
|
|
"description": "Read billing tests"
|
|
}
|
|
},
|
|
{
|
|
"tool": "Bash",
|
|
"input": {
|
|
"command": "grep -n -i \"coverage\\|^### Step\\|^## Step\" /tmp/gstack-paid-shard-6fJnEY/tmp/gstack-hermetic-3647161-TszbDZ/.claude/projects/-tmp-gstack-paid-shard-6fJnEY-tmp-skill-e2e-coverage-wQ70Nn/20dd6310-1b3c-4ff1-a914-f2ba87ad9bba/tool-results/bs0sgawur.txt | head -40",
|
|
"description": "Find coverage audit section in SKILL.md"
|
|
}
|
|
},
|
|
{
|
|
"tool": "Bash",
|
|
"input": {
|
|
"command": "cat /tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn/ship/sections/test-coverage.md 2>/dev/null || cat ~/.claude/skills/gstack/ship/sections/test-coverage.md 2>/dev/null || find / -name test-coverage.md -path '*ship*' 2>/dev/null | head -3",
|
|
"description": "Read the test-coverage section instructions"
|
|
}
|
|
},
|
|
{
|
|
"tool": "Bash",
|
|
"input": {
|
|
"command": "git diff main...HEAD --stat; echo \"---\"; git log --oneline main..HEAD; echo \"--- test files:\"; git ls-files | grep -E '(\\.test\\.|\\.spec\\.|_test\\.|_spec\\.)' | wc -l; echo \"--- CLAUDE.md:\"; cat CLAUDE.md 2>/dev/null | head -20; ls",
|
|
"description": "Check diff vs main, test file count, CLAUDE.md"
|
|
}
|
|
}
|
|
],
|
|
"durationMs": 68113,
|
|
"costUsd": 0.44,
|
|
"turnsUsed": 8,
|
|
"model": "claude-fable-5-1"
|
|
}
|
|
},
|
|
{
|
|
"attempt": 2,
|
|
"recordedPassed": true,
|
|
"cwd": "/tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn",
|
|
"sessionId": "deabdb19-102d-4986-a42d-2812387e2b8c",
|
|
"result": {
|
|
"exitReason": "success",
|
|
"browseErrors": [],
|
|
"output": "Everything needed is in hand: one source file, one test file, vitest detected from `vitest.config.ts`, and one test file tracked. The branch has no diff against main, since both point at the initial commit, so I audited the whole of `src/billing.ts` as the changed code. I ran the audit inline rather than dispatching a subagent because the scope is two small files and you asked for the diagram directly.\n\n```\nCODE PATHS USER FLOWS\n[+] src/billing.ts [+] Payment checkout\n \u251c\u2500\u2500 processPayment(amount, currency) \u251c\u2500\u2500 [GAP] [\u2192E2E] Complete purchase (USD/EUR) end to end\n \u2502 \u251c\u2500\u2500 [\u2605\u2605 TESTED] happy path, USD \u2014 billing.test.ts:6 \u251c\u2500\u2500 [GAP] Zero/negative amount error shown to user\n \u2502 \u251c\u2500\u2500 [GAP] happy path, EUR (2nd operand of &&) \u2514\u2500\u2500 [GAP] Unsupported currency error shown to user\n \u2502 \u251c\u2500\u2500 [GAP] amount <= 0 \u2192 'Invalid amount'\n \u2502 \u2502 (edge: amount = 0, negative, NaN*) [+] Refund\n \u2502 \u2514\u2500\u2500 [GAP] currency \u2209 {USD,EUR} \u2192 'Unsupported' \u251c\u2500\u2500 [GAP] [\u2192E2E] Full refund of an existing payment\n \u2502 (edge: lowercase 'usd', empty string) \u2514\u2500\u2500 [GAP] Refund submitted with empty reason\n \u2514\u2500\u2500 refundPayment(paymentId, reason)\n \u251c\u2500\u2500 [GAP] !paymentId \u2192 'Payment ID required'\n \u251c\u2500\u2500 [GAP] !reason \u2192 'Reason required'\n \u2514\u2500\u2500 [GAP] happy path \u2192 { status: 'refunded' }\n\n* NaN <= 0 is false, so processPayment(NaN, 'USD') returns success. Untested and likely a bug.\n\nLLM integration: none in diff \u2014 no eval needed\n\nCOVERAGE: 1/12 paths tested (8%) | Code paths: 1/7 (14%) | User flows: 0/5 (0%)\nQUALITY: \u2605\u2605\u2605:0 \u2605\u2605:1 \u2605:0 | GAPS: 11 (2 E2E, 0 eval)\n```\n\nLegend: \u2605\u2605\u2605 behavior + edge + error | \u2605\u2605 happy path | \u2605 smoke check\n[\u2192E2E] = needs integration test | [\u2192EVAL] = needs LLM eval\n\n**Summary**\n\n- **Coverage: 8%, 11 gaps. 0 tests added.** The only existing test checks the USD happy path of processPayment and asserts on status alone.\n- **Both error paths in processPayment are untested**, as is the whole of refundPayment. The test file's own comments already flag these three gaps.\n- **One latent bug surfaced.** A NaN amount passes the `amount <= 0` guard and returns success. An invalid-amount test should include this case.\n- **No regression.** There is no diff against main, so nothing previously working has been changed. The regression rule does not apply.\n- **Coverage gate would fail.** With no CLAUDE.md thresholds, the default minimum is 60%. The gate would prompt for a generation pass or an override. I did not prompt, since you scoped this run to the diagram only.\n\nTest generation and the test-plan artifact under `~/.gstack/projects` were skipped per your instructions.\n\nTests: 1 \u2192 1 (+0 new)\n\n```json\n{\"coverage_pct\":8,\"gaps\":11,\"diagram\":\"CODE PATHS / USER FLOWS diagram as shown above\",\"tests_added\":[]}\n```",
|
|
"transcript": [
|
|
{
|
|
"type": "system",
|
|
"subtype": "init",
|
|
"session_id": "deabdb19-102d-4986-a42d-2812387e2b8c",
|
|
"cwd": "/tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn"
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "deabdb19-102d-4986-a42d-2812387e2b8c",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"id": "msg_011CetqhT1EgvtVUV5mJJwvW",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_01FHUbjwyNijdGxesdG693W2",
|
|
"name": "Bash",
|
|
"input": {
|
|
"command": "ls; find . -name SKILL.md -path \"*ship*\" 2>/dev/null | head; ls ship 2>/dev/null",
|
|
"description": "Locate ship/SKILL.md and list project"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "deabdb19-102d-4986-a42d-2812387e2b8c",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_01FHUbjwyNijdGxesdG693W2",
|
|
"is_error": false,
|
|
"content": "CHANGELOG.md\nVERSION\npackage.json\nreview\nship\nsrc\ntest\nvitest.config.ts\n./ship/SKILL.md\nSKILL.md\nSKILL.md.tmpl\nsections"
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "deabdb19-102d-4986-a42d-2812387e2b8c",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"id": "msg_011CetqhxyHjQFA74N1eQm7e",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_015u5VirTkj1tUJEfFYVuFX6",
|
|
"name": "Bash",
|
|
"input": {
|
|
"command": "cd /tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn; grep -n \"3.4\" ship/SKILL.md | head; echo ---; awk '/Step 3.4|3\\.4/{f=1} f{print} /Step 3\\.5|### Step 4|## Step 4/{if(f)exit}' ship/SKILL.md | head -150",
|
|
"description": "Extract Step 3.4 from ship/SKILL.md"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "deabdb19-102d-4986-a42d-2812387e2b8c",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_015u5VirTkj1tUJEfFYVuFX6",
|
|
"is_error": false,
|
|
"content": "---"
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "deabdb19-102d-4986-a42d-2812387e2b8c",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"id": "msg_011CetqibZhdYz2CqaQrsKxs",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_018EJsYQDBF1RZCPhecEyuKd",
|
|
"name": "Bash",
|
|
"input": {
|
|
"command": "cd /tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn; wc -l ship/SKILL.md; grep -n -i \"coverage\\|^#\" ship/SKILL.md | head -60",
|
|
"description": "Find coverage audit section headings in SKILL.md"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "deabdb19-102d-4986-a42d-2812387e2b8c",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_018EJsYQDBF1RZCPhecEyuKd",
|
|
"is_error": false,
|
|
"content": "1124 ship/SKILL.md\n26:## When to invoke this skill\n33:## Preamble (run first)\n61:## Plan Mode Safe Operations\n65:## Skill Invocation During Plan Mode\n73:## AskUserQuestion Format\n75:### Tool resolution (read first)\n84:### When AskUserQuestion is unavailable or a call fails\n108:### Format\n118:Completeness: A=X/10, B=Y/10 (or: Note: options differ in kind, not coverage \u2014 no completeness score)\n133:Completeness: use `Completeness: N/10` only when options differ in coverage. 10 = complete, 7 = happy path, 3 = shortcut. If options differ in kind, write: `Note: options differ in kind, not coverage \u2014 no completeness score.`\n145:### Handling 5+ options \u2014 split, never drop\n168:### Self-check before emitting\n174:- [ ] Completeness scored (coverage) OR kind-note present (kind)\n186:## Artifacts Sync (skill start)\n197:## Model-Specific Behavioral Patch (claude)\n215:## Voice\n236:## Context Recovery\n269:## Writing Style (skip entirely if `EXPLAIN_LEVEL: terse` appears in the preamble echo OR the user's current message explicitly requests terse / no-explanations output)\n283:## Completeness Principle \u2014 Boil the Ocean\n285:AI makes completeness cheap, so the complete thing is the goal. Recommend full coverage (tests, edge cases, error paths) \u2014 boil the ocean one lake at a time. The only thing out of scope is genuinely unrelated work (rewrites, multi-quarter migrations); flag that as separate scope, never as an excuse for a shortcut.\n287:When options differ in coverage, include `Completeness: X/10` (10 = all edge cases, 7 = happy path, 3 = shortcut). When options differ in kind, write: `Note: options differ in kind, not coverage \u2014 no completeness score.` Do not fabricate scores.\n289:## Confusion Protocol\n293:## Claimed Limitations Need Evidence\n297:## Continuous Checkpoint Mode\n322:## Context Health (soft directive)\n328:## Question Tuning (skip entirely if `QUESTION_TUNING: false`)\n352:## Repo Ownership \u2014 See Something, Say Something\n360:## Search Before Building\n380:## Completion Status Protocol\n390:## Operational Self-Improvement\n406:## Telemetry (run last)\n427:## Plan Status Footer\n431:## Third-Party Web Actions\n459:## Step 0: Detect platform and base branch\n500:# Ship: Fully Automated Ship Workflow\n511:- AI-assessed coverage below target (see Step 7 for minimum/target decisions)\n525:- Test coverage gaps within target threshold (auto-generate and commit, or flag in PR body)\n529:(tests, coverage audit, plan completion, pre-landing review, adversarial review,\n539:## Section index \u2014 Read each section when its situation applies\n548:| auditing test coverage of the diff (Step 7) | `sections/test-coverage.md` |\n558:## Step 0.9: Apple target detection\n570:## Step 1: Pre-flight\n580:## Review Readiness Dashboard\n588:Render each record using its recorded host, source, outside_provider, outside_status, and phase. Historical source \"claude\" means a native Claude subagent; source \"claude-code\" means the external CLI. Never infer a historical provider from the current harness. Unknown model identity remains unknown. Missing/disabled/skipped outside coverage is distinct from native completion.\n594:Read `autoplan-voices` and `design-outside-voices` for the coverage detail below the dashboard. Group by workflow run and phase, not merely skill. Show each phase\u2019s recorded provider and outside_status; partial coverage must remain partial. These records do not change the engineering gate.\n619:- **Outside Voice (default-on):** Independent plan review through the host-selected provider after /plan-ceo-review and /plan-eng-review. The codex_reviews switch disables the entire extra step. Provider failure uses the existing native fallback and reports missing outside coverage. Never gates shipping.\n648:## Step 2: Distribution Pipeline Check\n676:## Step 3: Merge the base branch (BEFORE tests)\n693:> **STOP.** Before auditing test coverage of the diff (Step 7), Read `~/.claude/skills/gstack/ship/sections/test-coverage.md` and execute it\n708:## Step 12: Version bump (auto-decide)\n752:## Step 14: TODOS.md (auto-update)\n802:## Step 15: Commit (bisectable chunks)\n804:### Step 15.0: WIP Commit Squash (continuous checkpoint mode only)\n822:# Export [gstack-context] blocks from all WIP commits on this branch.\n823:# This file becomes input to the CHANGELOG entry and may inform PR body context.\n862:# Branch contains only WIP commits. Reset-soft is safe here because there's\n863:# nothing non-WIP to preserve. Verify first.\n881:### Step 15.1: Bisectable Commits\n919:## Step 16: Verification Gate\n962:## Step 17: Push"
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "deabdb19-102d-4986-a42d-2812387e2b8c",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"id": "msg_011CetqixBSKC9No3S4z6UmZ",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_01NZCsC3Q6mP9pb1jCPWb8Fy",
|
|
"name": "Bash",
|
|
"input": {
|
|
"command": "cd /tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn; cat ship/sections/test-coverage.md",
|
|
"description": "Read test-coverage section instructions"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "deabdb19-102d-4986-a42d-2812387e2b8c",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_01NZCsC3Q6mP9pb1jCPWb8Fy",
|
|
"is_error": false,
|
|
"content": "<!-- AUTO-GENERATED from test-coverage.md.tmpl \u2014 do not edit directly -->\n<!-- Regenerate: bun run gen:skill-docs -->\n## Step 7: Test Coverage Audit\n\n**Dispatch this step as a subagent** using the Agent tool with `subagent_type: \"general-purpose\"`. The subagent runs the coverage audit in a fresh context window \u2014 the parent only sees the conclusion, not intermediate file reads. This is context-rot defense.\n\n**Foreground required:** pass `run_in_background: false` on the Agent call \u2014 subagents run in the BACKGROUND by default since Claude Code v2.1.198. (Merely omitting the flag no longer produces a foreground run; it must be explicitly false.) The dispatch happens ONLY via the Agent tool: invoking the target as a Skill, or executing its workflow inline in your own context, is WRONG even though the skill may appear in your available-skills list \u2014 inline execution forfeits the fresh-context isolation this dispatch exists for, and the explicit flag already makes the Agent call block. (Where a step defines an inline FALLBACK, it applies only after a dispatched subagent has failed.) The parent needs this audit's LAST-line JSON before continuing.\n\n**Subagent prompt:** Pass the following instructions to the subagent, with `<base>` substituted with the base branch:\n\n````text\nYou are running a ship-workflow test coverage audit. Run `git diff <base>...HEAD` as needed. Do not commit or push. Perform only this audit; return unresolved user decisions to the parent instead of asking or advancing to another workflow step.\n\n100% coverage is the goal \u2014 every untested path is a path where bugs hide and vibe coding becomes yolo coding. Evaluate what was ACTUALLY coded (from the diff), not what was planned.\n\n### Test Framework Detection\n\nBefore analyzing coverage, detect the project's test framework:\n\n1. **Read CLAUDE.md** \u2014 look for a `## Testing` section with test command and framework name. If found, use that as the authoritative source.\n2. **If CLAUDE.md has no testing section, auto-detect:**\n\n```bash\nsetopt +o nomatch 2>/dev/null || true # zsh compat\n# Detect project runtime (markers are evidence, not commands to run blind)\n[ -f manage.py ] && echo \"RUNTIME:python FRAMEWORK:django\"\n{ [ -f pyproject.toml ] || [ -f pytest.ini ] || [ -f tox.ini ] || [ -f setup.cfg ] || [ -f requirements.txt ]; } && echo \"RUNTIME:python\"\n[ -f Gemfile ] || [ -f Rakefile ] || [ -f .rspec ] && echo \"RUNTIME:ruby\"\n[ -f package.json ] && echo \"RUNTIME:node\"\n[ -f go.mod ] && echo \"RUNTIME:go\"\n[ -f Cargo.toml ] && echo \"RUNTIME:rust\"\n[ -f pom.xml ] && echo \"RUNTIME:jvm BUILD:maven\"\n{ [ -f build.gradle ] || [ -f build.gradle.kts ]; } && echo \"RUNTIME:jvm BUILD:gradle\"\n# Check for existing test infrastructure \u2014 config files, scripts, AND test files\nls jest.config.* vitest.config.* playwright.config.* cypress.config.* .rspec pytest.ini tox.ini phpunit.xml 2>/dev/null\n[ -f package.json ] && grep -q '\"test\"[[:space:]]*:' package.json && echo \"SCRIPT:package.json test\"\n[ -f Makefile ] && grep -qE '^(test|check):' Makefile && echo \"TARGET:make test\"\ngit ls-files | grep -cE '(^|/)(tests?|spec|__tests__)/|(^|/)tests?\\.py$|(^|/)test_[^/]+\\.py$|_test\\.(go|py|rb|ts|js|exs)$|\\.(test|spec)\\.[jt]sx?$|_spec\\.rb$|Test\\.(java|kt)$' | sed 's/^/TESTFILES:/'\n```\n\n3. **If no framework detected:** use the bootstrap decision already made in Step 4; report diagram-only coverage if setup was declined. Do not restart bootstrap from this audit.\n\n**0. Before/after test count:**\n\n```bash\n# Count test files before any generation\ngit ls-files 2>/dev/null | grep -E '(\\.test\\.|\\.spec\\.|_test\\.|_spec\\.)' | wc -l\n```\n\nStore this number for the PR body.\n\n**1. Trace every codepath changed** using `git diff origin/<base>...HEAD`:\n\nRead every changed file. For each one, trace how data flows through the code \u2014 don't just list functions, actually follow the execution:\n\n1. **Read the diff.** For each changed file, read the full file (not just the diff hunk) to understand context.\n2. **Trace data flow.** Starting from each entry point (route handler, exported function, event listener, component render), follow the data through every branch:\n - Where does input come from? (request params, props, database, API call)\n - What transforms it? (validation, mapping, computation)\n - Where does it go? (database write, API response, rendered output, side effect)\n - What can go wrong at each step? (null/undefined, invalid input, network failure, empty collection)\n3. **Diagram the execution.** For each changed file, draw an ASCII diagram showing:\n - Every function/method that was added or modified\n - Every conditional branch (if/else, switch, ternary, guard clause, early return)\n - Every error path (try/catch, rescue, error boundary, fallback)\n - Every call to another function (trace into it \u2014 does IT have untested branches?)\n - Every edge: what happens with null input? Empty array? Invalid type?\n\nThis is the critical step \u2014 you're building a map of every line of code that can execute differently based on input. Every branch in this diagram needs a test.\n\n**2. Map user flows, interactions, and error states:**\n\nCode coverage isn't enough \u2014 you need to cover how real users interact with the changed code. For each changed feature, think through:\n\n- **User flows:** What sequence of actions does a user take that touches this code? Map the full journey (e.g., \"user clicks 'Pay' \u2192 form validates \u2192 API call \u2192 success/failure screen\"). Each step in the journey needs a test.\n- **Interaction edge cases:** What happens when the user does something unexpected?\n - Double-click/rapid resubmit\n - Navigate away mid-operation (back button, close tab, click another link)\n - Submit with stale data (page sat open for 30 minutes, session expired)\n - Slow connection (API takes 10 seconds \u2014 what does the user see?)\n - Concurrent actions (two tabs, same form)\n- **Error states the user can see:** For every error the code handles, what does the user actually experience?\n - Is there a clear error message or a silent failure?\n - Can the user recover (retry, go back, fix input) or are they stuck?\n - What happens with no network? With a 500 from the API? With invalid data from the server?\n- **Empty/zero/boundary states:** What does the UI show with zero results? With 10,000 results? With a single character input? With maximum-length input?\n\nAdd these to your diagram alongside the code branches. A user flow with no test is just as much a gap as an untested if/else.\n\n**3. Check each branch against existing tests:**\n\nGo through your diagram branch by branch \u2014 both code paths AND user flows. For each one, search for a test that exercises it:\n- Function `processPayment()` \u2192 look for `billing.test.ts`, `billing.spec.ts`, `test/billing_test.rb`\n- An if/else \u2192 look for tests covering BOTH the true AND false path\n- An error handler \u2192 look for a test that triggers that specific error condition\n- A call to `helperFn()` that has its own branches \u2192 those branches need tests too\n- A user flow \u2192 look for an integration or E2E test that walks through the journey\n- An interaction edge case \u2192 look for a test that simulates the unexpected action\n\nQuality scoring rubric:\n- \u2605\u2605\u2605 Tests behavior with edge cases AND error paths\n- \u2605\u2605 Tests correct behavior, happy path only\n- \u2605 Smoke test / existence check / trivial assertion (e.g., \"it renders\", \"it doesn't throw\")\n\n### E2E Test Decision Matrix\n\nWhen checking each branch, also determine whether a unit test or E2E/integration test is the right tool:\n\n**RECOMMEND E2E (mark as [\u2192E2E] in the diagram):**\n- Common user flow spanning 3+ components/services (e.g., signup \u2192 verify email \u2192 first login)\n- Integration point where mocking hides real failures (e.g., API \u2192 queue \u2192 worker \u2192 DB)\n- Auth/payment/data-destruction flows \u2014 too important to trust unit tests alone\n\n**RECOMMEND EVAL (mark as [\u2192EVAL] in the diagram):**\n- Critical LLM call that needs a quality eval (e.g., prompt change \u2192 test output still meets quality bar)\n- Changes to prompt templates, system instructions, or tool definitions\n\n**STICK WITH UNIT TESTS:**\n- Pure function with clear inputs/outputs\n- Internal helper with no side effects\n- Edge case of a single function (null input, empty array)\n- Obscure/rare flow that isn't customer-facing\n\n### REGRESSION RULE (mandatory)\n\n**IRON RULE:** When the coverage audit identifies a REGRESSION \u2014 code that previously worked but the diff broke \u2014 a regression test is written immediately. No AskUserQuestion. No skipping. Regressions are the highest-priority test because they prove something broke.\n\nA regression is when:\n- The diff modifies existing behavior (not new code)\n- The existing test suite (if any) doesn't cover the changed path\n- The change introduces a new failure mode for existing callers\n\nWhen uncertain whether a change is a regression, err on the side of writing the test.\n\n**4. Output ASCII coverage diagram:**\n\nInclude BOTH code paths and user flows in the same diagram. Mark E2E-worthy and eval-worthy paths:\n\n```\nCODE PATHS USER FLOWS\n[+] src/services/billing.ts [+] Payment checkout\n \u251c\u2500\u2500 processPayment() \u251c\u2500\u2500 [\u2605\u2605\u2605 TESTED] Complete purchase \u2014 checkout.e2e.ts:15\n \u2502 \u251c\u2500\u2500 [\u2605\u2605\u2605 TESTED] happy + declined + timeout \u251c\u2500\u2500 [GAP] [\u2192E2E] Double-click submit\n \u2502 \u251c\u2500\u2500 [GAP] Network timeout \u2514\u2500\u2500 [GAP] Navigate away mid-payment\n \u2502 \u2514\u2500\u2500 [GAP] Invalid currency\n \u2514\u2500\u2500 refundPayment() [+] Error states\n \u251c\u2500\u2500 [\u2605\u2605 TESTED] Full refund \u2014 :89 \u251c\u2500\u2500 [\u2605\u2605 TESTED] Card declined message\n \u2514\u2500\u2500 [\u2605 TESTED] Partial (non-throw only) \u2014 :101 \u2514\u2500\u2500 [GAP] Network timeout UX\n\nLLM integration: [GAP] [\u2192EVAL] Prompt template change \u2014 needs eval test\n\nCOVERAGE: 5/13 paths tested (38%) | Code paths: 3/5 (60%) | User flows: 2/8 (25%)\nQUALITY: \u2605\u2605\u2605:2 \u2605\u2605:2 \u2605:1 | GAPS: 8 (2 E2E, 1 eval)\n```\n\nLegend: \u2605\u2605\u2605 behavior + edge + error | \u2605\u2605 happy path | \u2605 smoke check\n[\u2192E2E] = needs integration test | [\u2192EVAL] = needs LLM eval\n\n**Fast path:** All paths covered \u2192 \"Step 7: All new code paths have test coverage \u2713\" Continue.\n\n**5. Generate tests for uncovered paths:**\n\nIf test framework detected (or bootstrapped in Step 4):\n- Prioritize error handlers and edge cases first (happy paths are more likely already tested)\n- Read 2-3 existing test files to match conventions exactly\n- Generate unit tests. Mock all external dependencies (DB, API, Redis).\n- For paths marked [\u2192E2E]: generate integration/E2E tests using the project's E2E framework (Playwright, Cypress, Capybara, etc.)\n- For paths marked [\u2192EVAL]: generate eval tests using the project's eval framework, or flag for manual eval if none exists\n- Write tests that exercise the specific uncovered path with real assertions\n- Run each test. Passes \u2192 keep the change and report its path; the parent commits in Step 15.\n- Fails \u2192 fix once. Still fails \u2192 revert, note gap in diagram.\n\nCaps: 30 code paths max, 20 tests generated max (code + user flow combined), 2-min per-test exploration cap.\n\nIf no test framework AND user declined bootstrap \u2192 diagram only, no generation. Note: \"Test generation skipped \u2014 no test framework configured.\"\n\n**Diff is test-only changes:** Return a skipped audit with null coverage, zero gaps, and \"No new application code paths to audit.\"\n\n**6. After-count and coverage summary:**\n\n```bash\n# Count test files after generation\ngit ls-files 2>/dev/null | grep -E '(\\.test\\.|\\.spec\\.|_test\\.|_spec\\.)' | wc -l\n```\n\nFor PR body: `Tests: {before} \u2192 {after} (+{delta} new)`\nCoverage line: `Test Coverage Audit: N new code paths. M covered (X%). K tests generated, awaiting parent commit.`\n\n### Test Plan Artifact\n\nAfter producing the coverage diagram, write a test plan artifact so `/qa` and `/qa-only` can consume it:\n\n```bash\neval \"$(~/.claude/skills/gstack/bin/gstack-slug 2>/dev/null)\" && mkdir -p ~/.gstack/projects/$SLUG\nUSER=$(whoami)\nDATETIME=$(date +%Y%m%d-%H%M%S)\n```\n\nWrite to `~/.gstack/projects/{slug}/{user}-{branch}-ship-test-plan-{datetime}.md`:\n\n```markdown\n# Test Plan\nGenerated by /ship on {date}\nBranch: {branch}\nRepo: {owner/repo}\n\n## Affected Pages/Routes\n- {URL path} \u2014 {what to test and why}\n\n## Key Interactions to Verify\n- {interaction description} on {page}\n\n## Edge Cases\n- {edge case} on {page}\n\n## Critical Paths\n- {end-to-end flow that must work}\n```\n\nAfter your analysis, output a single JSON object on the LAST LINE of your response (no other text after it):\n{\"coverage_pct\":N,\"gaps\":N,\"diagram\":\"<full markdown coverage diagram for PR body>\",\"tests_added\":[\"path\",...]}\nUse null for an undetermined or skipped coverage percentage, not zero. Include every remaining gap in the diagram so the parent can target a second pass.\n````\n\n**Parent processing:**\n\n1. Read the subagent's final output. Parse the LAST line as JSON.\n2. Store `coverage_pct` (for Step 20 metrics), `gaps` (user summary), `tests_added` (for the commit).\n3. Embed `diagram` verbatim in the PR body's `## Test Coverage` section (Step 19).\n4. Print a one-line summary: `Coverage: {coverage_pct}%, {gaps} gaps. {tests_added.length} tests added.`\n\n**If the subagent fails, times out, returns invalid JSON, or never completes (backgrounded despite the flag, or no final output after ~10 minutes \u2014 stop waiting; if a backgrounded task is still running, stop it first so a late result never races the fallback):** Fall back to running the audit inline in the parent. Do not block /ship on subagent failure \u2014 partial results are better than none.\n\n\n**7. Coverage gate:**\n\nThe parent owns this gate after receiving the audit result, including after an inline fallback. Generated tests stay uncommitted until Step 15. Any further generation uses the same audit prompt with the remaining gaps and pass count supplied.\n\nBefore proceeding, check CLAUDE.md for a `## Test Coverage` section with `Minimum:` and `Target:` fields. If found, use those percentages. Otherwise use defaults: Minimum = 60%, Target = 80%.\n\nUsing the coverage percentage from the diagram in substep 4 (the `COVERAGE: X/Y (Z%)` line):\n\n- **>= target:** Pass. \"Coverage gate: PASS ({X}%).\" Continue.\n- **>= minimum, < target:** Use AskUserQuestion:\n - \"AI-assessed coverage is {X}%. {N} code paths are untested. Target is {target}%.\"\n - RECOMMENDATION: Choose A because untested code paths are where production bugs hide.\n - Options:\n A) Generate more tests for remaining gaps (recommended)\n B) Ship anyway \u2014 I accept the coverage risk\n C) These paths don't need tests \u2014 mark as intentionally uncovered\n - If A: Dispatch one more generation pass targeting remaining gaps, then re-evaluate the result here. Maximum 2 generation passes total. At the cap, offer only B/C or stop; do not offer another generation pass.\n - If B: Continue. Include in PR body: \"Coverage gate: {X}% \u2014 user accepted risk.\"\n - If C: Continue. Include in PR body: \"Coverage gate: {X}% \u2014 {N} paths intentionally uncovered.\"\n\n- **< minimum:** Use AskUserQuestion:\n - \"AI-assessed coverage is critically low ({X}%). {N} of {M} code paths have no tests. Minimum threshold is {minimum}%.\"\n - RECOMMENDATION: Choose A because less than {minimum}% means more code is untested than tested.\n - Options:\n A) Generate tests for remaining gaps (recommended)\n B) Override \u2014 ship with low coverage (I understand the risk)\n - If A: Dispatch one more generation pass. Maximum 2 passes total. At the cap, offer only B or stop; do not offer another generation pass.\n - If B: Continue. Include in PR body: \"Coverage gate: OVERRIDDEN at {X}%.\"\n\n**Coverage percentage undetermined:** If the coverage diagram doesn't produce a clear numeric percentage (ambiguous output, parse error), **skip the gate** with: \"Coverage gate: could not determine percentage \u2014 skipping.\" Do not default to 0% or block.\n\n**Test-only diffs:** Skip the gate (same as the existing fast-path).\n\n**100% coverage:** \"Coverage gate: PASS (100%).\" Continue.\n\n---"
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "deabdb19-102d-4986-a42d-2812387e2b8c",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"id": "msg_011CetqjP88B446H38cAJ9Wf",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_01DrGeAaqyzjNt43ExELcKwH",
|
|
"name": "Bash",
|
|
"input": {
|
|
"command": "cd /tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn; echo \"=== SRC ===\"; cat -n src/billing.ts; echo; echo \"=== TEST ===\"; cat -n test/billing.test.ts; echo; echo \"=== DIFF main...HEAD ===\"; git diff main...HEAD --stat; echo; echo \"=== TESTFILES ===\"; git ls-files | grep -E '(\\.test\\.|\\.spec\\.|_test\\.|_spec\\.)' | wc -l; echo \"=== CLAUDE.md ===\"; cat CLAUDE.md 2>/dev/null || echo none; echo \"=== package.json ===\"; cat package.json",
|
|
"description": "Read source, tests, diff, test-file count, and config"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "deabdb19-102d-4986-a42d-2812387e2b8c",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_01DrGeAaqyzjNt43ExELcKwH",
|
|
"is_error": false,
|
|
"content": "=== SRC ===\n 1\t\n 2\texport function processPayment(amount: number, currency: string) {\n 3\t if (amount <= 0) throw new Error('Invalid amount');\n 4\t if (currency !== 'USD' && currency !== 'EUR') throw new Error('Unsupported currency');\n 5\t return { status: 'success', amount, currency };\n 6\t}\n 7\t\n 8\texport function refundPayment(paymentId: string, reason: string) {\n 9\t if (!paymentId) throw new Error('Payment ID required');\n 10\t if (!reason) throw new Error('Reason required');\n 11\t return { status: 'refunded', paymentId, reason };\n 12\t}\n\n=== TEST ===\n 1\t\n 2\timport { describe, test, expect } from 'vitest';\n 3\timport { processPayment } from '../src/billing';\n 4\t\n 5\tdescribe('processPayment', () => {\n 6\t test('processes valid payment', () => {\n 7\t const result = processPayment(100, 'USD');\n 8\t expect(result.status).toBe('success');\n 9\t });\n 10\t // GAP: no test for invalid amount\n 11\t // GAP: no test for unsupported currency\n 12\t // GAP: refundPayment not tested at all\n 13\t});\n\n=== DIFF main...HEAD ===\n\n=== TESTFILES ===\n1\n=== CLAUDE.md ===\nnone\n=== package.json ===\n{\n \"name\": \"test-coverage-app\",\n \"version\": \"1.0.0\",\n \"type\": \"module\",\n \"scripts\": {\n \"test\": \"echo \\\"no tests yet\\\"\"\n },\n \"devDependencies\": {\n \"vitest\": \"^1.0.0\"\n }\n}"
|
|
}
|
|
]
|
|
}
|
|
}
|
|
],
|
|
"toolCalls": [
|
|
{
|
|
"tool": "Bash",
|
|
"input": {
|
|
"command": "ls; find . -name SKILL.md -path \"*ship*\" 2>/dev/null | head; ls ship 2>/dev/null",
|
|
"description": "Locate ship/SKILL.md and list project"
|
|
}
|
|
},
|
|
{
|
|
"tool": "Bash",
|
|
"input": {
|
|
"command": "cd /tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn; grep -n \"3.4\" ship/SKILL.md | head; echo ---; awk '/Step 3.4|3\\.4/{f=1} f{print} /Step 3\\.5|### Step 4|## Step 4/{if(f)exit}' ship/SKILL.md | head -150",
|
|
"description": "Extract Step 3.4 from ship/SKILL.md"
|
|
}
|
|
},
|
|
{
|
|
"tool": "Bash",
|
|
"input": {
|
|
"command": "cd /tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn; wc -l ship/SKILL.md; grep -n -i \"coverage\\|^#\" ship/SKILL.md | head -60",
|
|
"description": "Find coverage audit section headings in SKILL.md"
|
|
}
|
|
},
|
|
{
|
|
"tool": "Bash",
|
|
"input": {
|
|
"command": "cd /tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn; cat ship/sections/test-coverage.md",
|
|
"description": "Read test-coverage section instructions"
|
|
}
|
|
},
|
|
{
|
|
"tool": "Bash",
|
|
"input": {
|
|
"command": "cd /tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn; echo \"=== SRC ===\"; cat -n src/billing.ts; echo; echo \"=== TEST ===\"; cat -n test/billing.test.ts; echo; echo \"=== DIFF main...HEAD ===\"; git diff main...HEAD --stat; echo; echo \"=== TESTFILES ===\"; git ls-files | grep -E '(\\.test\\.|\\.spec\\.|_test\\.|_spec\\.)' | wc -l; echo \"=== CLAUDE.md ===\"; cat CLAUDE.md 2>/dev/null || echo none; echo \"=== package.json ===\"; cat package.json",
|
|
"description": "Read source, tests, diff, test-file count, and config"
|
|
}
|
|
}
|
|
],
|
|
"durationMs": 72397,
|
|
"costUsd": 0.35,
|
|
"turnsUsed": 6,
|
|
"model": "claude-fable-5-1"
|
|
}
|
|
},
|
|
{
|
|
"attempt": 3,
|
|
"recordedPassed": false,
|
|
"cwd": "/tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn",
|
|
"sessionId": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"result": {
|
|
"exitReason": "timeout",
|
|
"browseErrors": [],
|
|
"output": "",
|
|
"transcript": [
|
|
{
|
|
"type": "system",
|
|
"subtype": "init",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"cwd": "/tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn"
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"id": "msg_011CetqnmnE64xgM7cy5EH9i",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_01WMPUAwNoNg6zvRSjQ9GFRv",
|
|
"name": "Bash",
|
|
"input": {
|
|
"command": "ls -la && find . -name \"SKILL.md\" -not -path \"./node_modules/*\" 2>/dev/null",
|
|
"description": "List project and locate SKILL.md"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_01WMPUAwNoNg6zvRSjQ9GFRv",
|
|
"is_error": false,
|
|
"content": "total 16\ndrwx------ 7 vercel-sandbox vercel-sandbox 146 Sep 10 00:48 .\ndrwxr-xr-x 6 vercel-sandbox vercel-sandbox 112 Sep 10 00:50 ..\ndrwxr-xr-x 7 vercel-sandbox vercel-sandbox 150 Sep 10 00:48 .git\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 12 Sep 10 00:48 CHANGELOG.md\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 8 Sep 10 00:48 VERSION\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 182 Sep 10 00:48 package.json\ndrwxr-xr-x 4 vercel-sandbox vercel-sandbox 174 Sep 10 00:48 review\ndrwxr-xr-x 3 vercel-sandbox vercel-sandbox 59 Sep 10 00:48 ship\ndrwxr-xr-x 2 vercel-sandbox vercel-sandbox 24 Sep 10 00:48 src\ndrwxr-xr-x 2 vercel-sandbox vercel-sandbox 29 Sep 10 00:48 test\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 89 Sep 10 00:48 vitest.config.ts\n./ship/SKILL.md\n./review/SKILL.md"
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"id": "msg_011CetqoGRAGejtZqNaaw2S1",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_01BW3xWnZnXZrUvi5cFd9rfM",
|
|
"name": "Bash",
|
|
"input": {
|
|
"command": "cat ship/SKILL.md",
|
|
"description": "Read ship skill instructions"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"id": "msg_011CetqoGRAGejtZqNaaw2S1",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_01A6j2arvuXuNzBwAX2EHFXg",
|
|
"name": "Bash",
|
|
"input": {
|
|
"command": "cat -n src/billing.ts",
|
|
"description": "Read billing source"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_01A6j2arvuXuNzBwAX2EHFXg",
|
|
"is_error": false,
|
|
"content": " 1\t\n 2\texport function processPayment(amount: number, currency: string) {\n 3\t if (amount <= 0) throw new Error('Invalid amount');\n 4\t if (currency !== 'USD' && currency !== 'EUR') throw new Error('Unsupported currency');\n 5\t return { status: 'success', amount, currency };\n 6\t}\n 7\t\n 8\texport function refundPayment(paymentId: string, reason: string) {\n 9\t if (!paymentId) throw new Error('Payment ID required');\n 10\t if (!reason) throw new Error('Reason required');\n 11\t return { status: 'refunded', paymentId, reason };\n 12\t}"
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_01BW3xWnZnXZrUvi5cFd9rfM",
|
|
"is_error": false,
|
|
"content": "<persisted-output>\nOutput too large (76.1KB). Full output saved to: /tmp/gstack-paid-shard-6fJnEY/tmp/gstack-hermetic-3647161-TszbDZ/.claude/projects/-tmp-gstack-paid-shard-6fJnEY-tmp-skill-e2e-coverage-wQ70Nn/034c10a5-6b9c-49cd-8e3e-1759bce3d453/tool-results/bk12cnf6b.txt\n\nPreview (first 2KB):\n---\nname: ship\npreamble-tier: 4\nversion: 1.0.0\ndescription: \"Ship workflow: detect + merge base branch, run tests, review diff, bump VERSION, update CHANGELOG, commit, push, create PR. (gstack)\"\nallowed-tools:\n - Bash\n - Read\n - Write\n - Edit\n - Grep\n - Glob\n - Agent\n - AskUserQuestion\n - WebSearch\ntriggers:\n - ship it\n - create a pr\n - push to main\n - deploy this\n---\n<!-- AUTO-GENERATED from SKILL.md.tmpl \u2014 do not edit directly -->\n<!-- Regenerate: bun run gen:skill-docs -->\n\n\n## When to invoke this skill\n\nUse when asked to \"ship\", \"deploy\",\n\"push to main\", \"create a PR\", \"merge and push\", or \"get it deployed\".\nProactively invoke this skill (do NOT push/PR directly) when the user says code\nis ready, asks about deploying, wants to push code up, or asks to create a PR.\n\n## Preamble (run first)\n\n```bash\n_SS=\"$HOME/.claude/skills/gstack/bin/gstack-skill-start\"\n[ -x \"$_SS\" ] || _SS=\".claude/skills/gstack/bin/gstack-skill-start\"\n\"$_SS\" --skill \"ship\" --model \"claude\" --parent-pid \"$PPID\" \\\n || echo \"SKILL_START: unavailable \u2014 stale install; run ./setup or /gstack-upgrade (preamble degraded, continue the user's task)\"\n```\n\nRead the echoed `KEY: value` STATUS lines \u2014 they drive every preamble rule\nbelow. **Degraded mode:** if `SKILL_START_PROTO: 1` is missing from the output\n(script absent, stale install, or a different protocol number), apply safe\ndefaults: treat `SESSION_KIND` as `interactive`, do NOT assume Conductor,\nskip onboarding/telemetry steps (their gates are marker-based, so consent and\nonboarding prompts are DEFERRED to the next healthy run \u2014 never lost), tell\nthe user to run `./setup` or `/gstack-upgrade`, and proceed with their task.\nNote `SESSION_ID` and `TEL_START` from the output \u2014 the Telemetry step needs\nthem at skill end.\n\n**Instruction blocks:** the output may contain\n`GSTACK_INSTRUCTION_BEGIN: <id> <session-id>` \u2026 `GSTACK_INSTRUCTION_END`\nblocks \u2014 one-time onboarding and consent directives whose runtime gates fired.\n...\n</persisted-output>"
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"id": "msg_011CetqoGRAGejtZqNaaw2S1",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_011R5H4HQWdHC4QvqNt7knN5",
|
|
"name": "Bash",
|
|
"input": {
|
|
"command": "cat -n test/billing.test.ts",
|
|
"description": "Read billing tests"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_011R5H4HQWdHC4QvqNt7knN5",
|
|
"is_error": false,
|
|
"content": " 1\t\n 2\timport { describe, test, expect } from 'vitest';\n 3\timport { processPayment } from '../src/billing';\n 4\t\n 5\tdescribe('processPayment', () => {\n 6\t test('processes valid payment', () => {\n 7\t const result = processPayment(100, 'USD');\n 8\t expect(result.status).toBe('success');\n 9\t });\n 10\t // GAP: no test for invalid amount\n 11\t // GAP: no test for unsupported currency\n 12\t // GAP: refundPayment not tested at all\n 13\t});"
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"id": "msg_011CetqoGRAGejtZqNaaw2S1",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_01AAFqHx8qvo8Qu19HqFRT4R",
|
|
"name": "Bash",
|
|
"input": {
|
|
"command": "git log --oneline main..HEAD; echo \"---diff vs main---\"; git diff main --stat; echo \"---\"; cat package.json",
|
|
"description": "Show branch diff vs main and package.json"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_01AAFqHx8qvo8Qu19HqFRT4R",
|
|
"is_error": false,
|
|
"content": "---diff vs main---\n---\n{\n \"name\": \"test-coverage-app\",\n \"version\": \"1.0.0\",\n \"type\": \"module\",\n \"scripts\": {\n \"test\": \"echo \\\"no tests yet\\\"\"\n },\n \"devDependencies\": {\n \"vitest\": \"^1.0.0\"\n }\n}"
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"id": "msg_011Cetqom1NxDd5CnNdssdei",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_01UvFFDmGakAvMa944ArzRw6",
|
|
"name": "Bash",
|
|
"input": {
|
|
"command": "grep -n -i \"3\\.4\\|coverage audit\\|Test Coverage\" ship/SKILL.md | head -30",
|
|
"description": "Locate Step 3.4 in SKILL.md"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_01UvFFDmGakAvMa944ArzRw6",
|
|
"is_error": false,
|
|
"content": "525:- Test coverage gaps within target threshold (auto-generate and commit, or flag in PR body)\n529:(tests, coverage audit, plan completion, pre-landing review, adversarial review,\n548:| auditing test coverage of the diff (Step 7) | `sections/test-coverage.md` |\n693:> **STOP.** Before auditing test coverage of the diff (Step 7), Read `~/.claude/skills/gstack/ship/sections/test-coverage.md` and execute it"
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"id": "msg_011CetqpAcwNazubHBKuByy2",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_01SrNwuaT6jEDC3BwngfHjTz",
|
|
"name": "Bash",
|
|
"input": {
|
|
"command": "sed -n 515,560p ship/SKILL.md; echo \"=== 680-720 ===\"; sed -n 680,720p ship/SKILL.md; echo \"=== sections dir ===\"; ls -la ship/sections 2>/dev/null; ls ~/.claude/skills/gstack/ship/sections/ 2>/dev/null",
|
|
"description": "Read coverage-audit step and locate sections file"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_01SrNwuaT6jEDC3BwngfHjTz",
|
|
"is_error": false,
|
|
"content": "- TODOS.md disorganized and user wants to reorganize (ask \u2014 see Step 14)\n\n**Never stop for:**\n- Uncommitted changes (always include them)\n- Version bump choice (auto-pick MICRO or PATCH \u2014 see Step 12)\n- CHANGELOG content (auto-generate from diff)\n- Commit message approval (auto-commit)\n- Multi-file changesets (auto-split into bisectable commits)\n- TODOS.md completed-item detection (auto-mark)\n- Auto-fixable review findings (dead code, N+1, stale comments \u2014 fixed automatically)\n- Test coverage gaps within target threshold (auto-generate and commit, or flag in PR body)\n\n**Re-run behavior (idempotency):**\nRe-running `/ship` means \"run the whole checklist again.\" Every verification step\n(tests, coverage audit, plan completion, pre-landing review, adversarial review,\nVERSION/CHANGELOG check, TODOS, document-release) runs on every invocation.\nOnly *actions* are idempotent:\n- Step 12: If VERSION already bumped, skip the bump but still read the version\n- Step 17: If already pushed, skip the push command\n- Step 19: If PR exists, update the body instead of creating a new PR\nNever skip a verification step because a prior `/ship` run already performed it.\n\n---\n\n## Section index \u2014 Read each section when its situation applies\n\nThis skill is a decision-tree skeleton. The steps below point to on-demand\nsections. Read a section in full before doing its step; do not work from memory.\n\n| When | Read this section |\n|------|-------------------|\n| the ship target is an Apple platform app (.xcodeproj, .xcworkspace, or an app-product Swift package) \u2014 read BEFORE Step 1's branch gate and any preflight; store distribution never routes through the branch/PR ceremony | `sections/apple-release.md` |\n| running the test suites and (if prompt files changed) the eval suites (Steps 4-6) | `sections/tests.md` |\n| auditing test coverage of the diff (Step 7) | `sections/test-coverage.md` |\n| auditing plan completion, verification, and scope drift (Step 8) | `sections/plan-completion.md` |\n| the pre-landing review and specialist dispatch (Step 9) | `sections/review-army.md` |\n| addressing Greptile review comments when a PR exists (Step 10) | `sections/greptile.md` |\n| the adversarial review and learnings capture (Step 11) | `sections/adversarial.md` |\n| writing the CHANGELOG entry (Step 13) | `sections/changelog.md` |\n| dispatching the /document-release subagent to sync docs (Step 18) and then creating or updating the PR/MR (Step 19) | `sections/pr-body.md` |\n\n---\n\n## Step 0.9: Apple target detection\n\nShipping to the App Store is not landing a PR. If the repository contains an\n=== 680-720 ===\n```bash\ngit fetch origin <base> && git merge origin/<base> --no-edit\n```\n\n**If there are merge conflicts:** Try to auto-resolve if they are simple (VERSION, schema.rb, CHANGELOG ordering). If conflicts are complex or ambiguous, **STOP** and show them.\n\n**If already up to date:** Continue silently.\n\n---\n\n> **STOP.** Before running the test suites and (if prompt files changed) the eval suites (Steps 4-6), Read `~/.claude/skills/gstack/ship/sections/tests.md` and execute it\n> in full. Do not work from memory \u2014 that section is the source of truth for this step.\n\n> **STOP.** Before auditing test coverage of the diff (Step 7), Read `~/.claude/skills/gstack/ship/sections/test-coverage.md` and execute it\n> in full. Do not work from memory \u2014 that section is the source of truth for this step.\n\n> **STOP.** Before auditing plan completion, verification, and scope drift (Step 8), Read `~/.claude/skills/gstack/ship/sections/plan-completion.md` and execute it\n> in full. Do not work from memory \u2014 that section is the source of truth for this step.\n\n> **STOP.** Before the pre-landing review and specialist dispatch (Step 9), Read `~/.claude/skills/gstack/ship/sections/review-army.md` and execute it\n> in full. Do not work from memory \u2014 that section is the source of truth for this step.\n\n> **STOP.** Before addressing Greptile review comments when a PR exists (Step 10), Read `~/.claude/skills/gstack/ship/sections/greptile.md` and execute it\n> in full. Do not work from memory \u2014 that section is the source of truth for this step.\n\n> **STOP.** Before the adversarial review and learnings capture (Step 11), Read `~/.claude/skills/gstack/ship/sections/adversarial.md` and execute it\n> in full. Do not work from memory \u2014 that section is the source of truth for this step.\n\n## Step 12: Version bump (auto-decide)\n\nUse **`gstack-version-bump`** for classify/write/repair and `gstack-next-version`\nfor slot selection. Bump level and queue collisions remain agent decisions.\n\n1. **Classify state** \u2014 pure reader, never writes:\n ```bash\n bun run ~/.claude/skills/gstack/bin/gstack-version-bump classify --base <base>\n ```\n Save the JSON `baseVersion` as `BASE_VERSION`, then read `state` and dispatch:\n - **FRESH** \u2192 do the bump (steps 2-4).\n - **ALREADY_BUMPED** \u2192 skip the bump, but run the queue-drift check (step 3) with the reported `currentVersion`. If the queue moved (next free version differs), **AskUserQuestion**: rebump to the new version (rewrites CHANGELOG header + PR title) or keep current (CI version-gate will reject until resolved).\n - **DRIFT_STALE_PKG** \u2192 run `gstack-version-bump repair` (syncs package.json to VERSION). No re-bump; reuse `currentVersion` for CHANGELOG + PR.\n=== sections dir ===\ntotal 260\ndrwxr-xr-x 2 vercel-sandbox vercel-sandbox 4096 Sep 10 00:48 .\ndrwxr-xr-x 3 vercel-sandbox vercel-sandbox 59 Sep 10 00:48 ..\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 22367 Sep 10 00:48 adversarial.md\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 1217 Sep 10 00:48 adversarial.md.tmpl\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 17059 Sep 10 00:48 apple-release.md\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 16939 Sep 10 00:48 apple-release.md.tmpl\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 1869 Sep 10 00:48 changelog.md\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 28 Sep 10 00:48 changelog.md.tmpl\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 4870 Sep 10 00:48 greptile.md\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 4096 Sep 10 00:48 greptile.md.tmpl\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 2515 Sep 10 00:48 manifest.json\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 22609 Sep 10 00:48 plan-completion.md\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 2785 Sep 10 00:48 plan-completion.md.tmpl\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 21176 Sep 10 00:48 pr-body.md\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 20403 Sep 10 00:48 pr-body.md.tmpl\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 30811 Sep 10 00:48 review-army.md\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 4102 Sep 10 00:48 review-army.md.tmpl\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 16290 Sep 10 00:48 test-coverage.md\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 1987 Sep 10 00:48 test-coverage.md.tmpl\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 22427 Sep 10 00:48 tests.md\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 6080 Sep 10 00:48 tests.md.tmpl\nadversarial.md\nadversarial.md.tmpl\napple-release.md\napple-release.md.tmpl\nchangelog.md\nchangelog.md.tmpl\ngreptile.md\ngreptile.md.tmpl\nmanifest.json\nplan-completion.md\nplan-completion.md.tmpl\npr-body.md\npr-body.md.tmpl\nreview-army.md\nreview-army.md.tmpl\ntest-coverage.md\ntest-coverage.md.tmpl\ntests.md\ntests.md.tmpl"
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"id": "msg_011CetqpAcwNazubHBKuByy2",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_0142tW5qRSKeTJnsnSZNQNu4",
|
|
"name": "Bash",
|
|
"input": {
|
|
"command": "cat ship/sections/test-coverage.md 2>/dev/null || cat ~/.claude/skills/gstack/ship/sections/test-coverage.md 2>/dev/null || echo \"NOT FOUND\"",
|
|
"description": "Read test-coverage section"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_0142tW5qRSKeTJnsnSZNQNu4",
|
|
"is_error": false,
|
|
"content": "<!-- AUTO-GENERATED from test-coverage.md.tmpl \u2014 do not edit directly -->\n<!-- Regenerate: bun run gen:skill-docs -->\n## Step 7: Test Coverage Audit\n\n**Dispatch this step as a subagent** using the Agent tool with `subagent_type: \"general-purpose\"`. The subagent runs the coverage audit in a fresh context window \u2014 the parent only sees the conclusion, not intermediate file reads. This is context-rot defense.\n\n**Foreground required:** pass `run_in_background: false` on the Agent call \u2014 subagents run in the BACKGROUND by default since Claude Code v2.1.198. (Merely omitting the flag no longer produces a foreground run; it must be explicitly false.) The dispatch happens ONLY via the Agent tool: invoking the target as a Skill, or executing its workflow inline in your own context, is WRONG even though the skill may appear in your available-skills list \u2014 inline execution forfeits the fresh-context isolation this dispatch exists for, and the explicit flag already makes the Agent call block. (Where a step defines an inline FALLBACK, it applies only after a dispatched subagent has failed.) The parent needs this audit's LAST-line JSON before continuing.\n\n**Subagent prompt:** Pass the following instructions to the subagent, with `<base>` substituted with the base branch:\n\n````text\nYou are running a ship-workflow test coverage audit. Run `git diff <base>...HEAD` as needed. Do not commit or push. Perform only this audit; return unresolved user decisions to the parent instead of asking or advancing to another workflow step.\n\n100% coverage is the goal \u2014 every untested path is a path where bugs hide and vibe coding becomes yolo coding. Evaluate what was ACTUALLY coded (from the diff), not what was planned.\n\n### Test Framework Detection\n\nBefore analyzing coverage, detect the project's test framework:\n\n1. **Read CLAUDE.md** \u2014 look for a `## Testing` section with test command and framework name. If found, use that as the authoritative source.\n2. **If CLAUDE.md has no testing section, auto-detect:**\n\n```bash\nsetopt +o nomatch 2>/dev/null || true # zsh compat\n# Detect project runtime (markers are evidence, not commands to run blind)\n[ -f manage.py ] && echo \"RUNTIME:python FRAMEWORK:django\"\n{ [ -f pyproject.toml ] || [ -f pytest.ini ] || [ -f tox.ini ] || [ -f setup.cfg ] || [ -f requirements.txt ]; } && echo \"RUNTIME:python\"\n[ -f Gemfile ] || [ -f Rakefile ] || [ -f .rspec ] && echo \"RUNTIME:ruby\"\n[ -f package.json ] && echo \"RUNTIME:node\"\n[ -f go.mod ] && echo \"RUNTIME:go\"\n[ -f Cargo.toml ] && echo \"RUNTIME:rust\"\n[ -f pom.xml ] && echo \"RUNTIME:jvm BUILD:maven\"\n{ [ -f build.gradle ] || [ -f build.gradle.kts ]; } && echo \"RUNTIME:jvm BUILD:gradle\"\n# Check for existing test infrastructure \u2014 config files, scripts, AND test files\nls jest.config.* vitest.config.* playwright.config.* cypress.config.* .rspec pytest.ini tox.ini phpunit.xml 2>/dev/null\n[ -f package.json ] && grep -q '\"test\"[[:space:]]*:' package.json && echo \"SCRIPT:package.json test\"\n[ -f Makefile ] && grep -qE '^(test|check):' Makefile && echo \"TARGET:make test\"\ngit ls-files | grep -cE '(^|/)(tests?|spec|__tests__)/|(^|/)tests?\\.py$|(^|/)test_[^/]+\\.py$|_test\\.(go|py|rb|ts|js|exs)$|\\.(test|spec)\\.[jt]sx?$|_spec\\.rb$|Test\\.(java|kt)$' | sed 's/^/TESTFILES:/'\n```\n\n3. **If no framework detected:** use the bootstrap decision already made in Step 4; report diagram-only coverage if setup was declined. Do not restart bootstrap from this audit.\n\n**0. Before/after test count:**\n\n```bash\n# Count test files before any generation\ngit ls-files 2>/dev/null | grep -E '(\\.test\\.|\\.spec\\.|_test\\.|_spec\\.)' | wc -l\n```\n\nStore this number for the PR body.\n\n**1. Trace every codepath changed** using `git diff origin/<base>...HEAD`:\n\nRead every changed file. For each one, trace how data flows through the code \u2014 don't just list functions, actually follow the execution:\n\n1. **Read the diff.** For each changed file, read the full file (not just the diff hunk) to understand context.\n2. **Trace data flow.** Starting from each entry point (route handler, exported function, event listener, component render), follow the data through every branch:\n - Where does input come from? (request params, props, database, API call)\n - What transforms it? (validation, mapping, computation)\n - Where does it go? (database write, API response, rendered output, side effect)\n - What can go wrong at each step? (null/undefined, invalid input, network failure, empty collection)\n3. **Diagram the execution.** For each changed file, draw an ASCII diagram showing:\n - Every function/method that was added or modified\n - Every conditional branch (if/else, switch, ternary, guard clause, early return)\n - Every error path (try/catch, rescue, error boundary, fallback)\n - Every call to another function (trace into it \u2014 does IT have untested branches?)\n - Every edge: what happens with null input? Empty array? Invalid type?\n\nThis is the critical step \u2014 you're building a map of every line of code that can execute differently based on input. Every branch in this diagram needs a test.\n\n**2. Map user flows, interactions, and error states:**\n\nCode coverage isn't enough \u2014 you need to cover how real users interact with the changed code. For each changed feature, think through:\n\n- **User flows:** What sequence of actions does a user take that touches this code? Map the full journey (e.g., \"user clicks 'Pay' \u2192 form validates \u2192 API call \u2192 success/failure screen\"). Each step in the journey needs a test.\n- **Interaction edge cases:** What happens when the user does something unexpected?\n - Double-click/rapid resubmit\n - Navigate away mid-operation (back button, close tab, click another link)\n - Submit with stale data (page sat open for 30 minutes, session expired)\n - Slow connection (API takes 10 seconds \u2014 what does the user see?)\n - Concurrent actions (two tabs, same form)\n- **Error states the user can see:** For every error the code handles, what does the user actually experience?\n - Is there a clear error message or a silent failure?\n - Can the user recover (retry, go back, fix input) or are they stuck?\n - What happens with no network? With a 500 from the API? With invalid data from the server?\n- **Empty/zero/boundary states:** What does the UI show with zero results? With 10,000 results? With a single character input? With maximum-length input?\n\nAdd these to your diagram alongside the code branches. A user flow with no test is just as much a gap as an untested if/else.\n\n**3. Check each branch against existing tests:**\n\nGo through your diagram branch by branch \u2014 both code paths AND user flows. For each one, search for a test that exercises it:\n- Function `processPayment()` \u2192 look for `billing.test.ts`, `billing.spec.ts`, `test/billing_test.rb`\n- An if/else \u2192 look for tests covering BOTH the true AND false path\n- An error handler \u2192 look for a test that triggers that specific error condition\n- A call to `helperFn()` that has its own branches \u2192 those branches need tests too\n- A user flow \u2192 look for an integration or E2E test that walks through the journey\n- An interaction edge case \u2192 look for a test that simulates the unexpected action\n\nQuality scoring rubric:\n- \u2605\u2605\u2605 Tests behavior with edge cases AND error paths\n- \u2605\u2605 Tests correct behavior, happy path only\n- \u2605 Smoke test / existence check / trivial assertion (e.g., \"it renders\", \"it doesn't throw\")\n\n### E2E Test Decision Matrix\n\nWhen checking each branch, also determine whether a unit test or E2E/integration test is the right tool:\n\n**RECOMMEND E2E (mark as [\u2192E2E] in the diagram):**\n- Common user flow spanning 3+ components/services (e.g., signup \u2192 verify email \u2192 first login)\n- Integration point where mocking hides real failures (e.g., API \u2192 queue \u2192 worker \u2192 DB)\n- Auth/payment/data-destruction flows \u2014 too important to trust unit tests alone\n\n**RECOMMEND EVAL (mark as [\u2192EVAL] in the diagram):**\n- Critical LLM call that needs a quality eval (e.g., prompt change \u2192 test output still meets quality bar)\n- Changes to prompt templates, system instructions, or tool definitions\n\n**STICK WITH UNIT TESTS:**\n- Pure function with clear inputs/outputs\n- Internal helper with no side effects\n- Edge case of a single function (null input, empty array)\n- Obscure/rare flow that isn't customer-facing\n\n### REGRESSION RULE (mandatory)\n\n**IRON RULE:** When the coverage audit identifies a REGRESSION \u2014 code that previously worked but the diff broke \u2014 a regression test is written immediately. No AskUserQuestion. No skipping. Regressions are the highest-priority test because they prove something broke.\n\nA regression is when:\n- The diff modifies existing behavior (not new code)\n- The existing test suite (if any) doesn't cover the changed path\n- The change introduces a new failure mode for existing callers\n\nWhen uncertain whether a change is a regression, err on the side of writing the test.\n\n**4. Output ASCII coverage diagram:**\n\nInclude BOTH code paths and user flows in the same diagram. Mark E2E-worthy and eval-worthy paths:\n\n```\nCODE PATHS USER FLOWS\n[+] src/services/billing.ts [+] Payment checkout\n \u251c\u2500\u2500 processPayment() \u251c\u2500\u2500 [\u2605\u2605\u2605 TESTED] Complete purchase \u2014 checkout.e2e.ts:15\n \u2502 \u251c\u2500\u2500 [\u2605\u2605\u2605 TESTED] happy + declined + timeout \u251c\u2500\u2500 [GAP] [\u2192E2E] Double-click submit\n \u2502 \u251c\u2500\u2500 [GAP] Network timeout \u2514\u2500\u2500 [GAP] Navigate away mid-payment\n \u2502 \u2514\u2500\u2500 [GAP] Invalid currency\n \u2514\u2500\u2500 refundPayment() [+] Error states\n \u251c\u2500\u2500 [\u2605\u2605 TESTED] Full refund \u2014 :89 \u251c\u2500\u2500 [\u2605\u2605 TESTED] Card declined message\n \u2514\u2500\u2500 [\u2605 TESTED] Partial (non-throw only) \u2014 :101 \u2514\u2500\u2500 [GAP] Network timeout UX\n\nLLM integration: [GAP] [\u2192EVAL] Prompt template change \u2014 needs eval test\n\nCOVERAGE: 5/13 paths tested (38%) | Code paths: 3/5 (60%) | User flows: 2/8 (25%)\nQUALITY: \u2605\u2605\u2605:2 \u2605\u2605:2 \u2605:1 | GAPS: 8 (2 E2E, 1 eval)\n```\n\nLegend: \u2605\u2605\u2605 behavior + edge + error | \u2605\u2605 happy path | \u2605 smoke check\n[\u2192E2E] = needs integration test | [\u2192EVAL] = needs LLM eval\n\n**Fast path:** All paths covered \u2192 \"Step 7: All new code paths have test coverage \u2713\" Continue.\n\n**5. Generate tests for uncovered paths:**\n\nIf test framework detected (or bootstrapped in Step 4):\n- Prioritize error handlers and edge cases first (happy paths are more likely already tested)\n- Read 2-3 existing test files to match conventions exactly\n- Generate unit tests. Mock all external dependencies (DB, API, Redis).\n- For paths marked [\u2192E2E]: generate integration/E2E tests using the project's E2E framework (Playwright, Cypress, Capybara, etc.)\n- For paths marked [\u2192EVAL]: generate eval tests using the project's eval framework, or flag for manual eval if none exists\n- Write tests that exercise the specific uncovered path with real assertions\n- Run each test. Passes \u2192 keep the change and report its path; the parent commits in Step 15.\n- Fails \u2192 fix once. Still fails \u2192 revert, note gap in diagram.\n\nCaps: 30 code paths max, 20 tests generated max (code + user flow combined), 2-min per-test exploration cap.\n\nIf no test framework AND user declined bootstrap \u2192 diagram only, no generation. Note: \"Test generation skipped \u2014 no test framework configured.\"\n\n**Diff is test-only changes:** Return a skipped audit with null coverage, zero gaps, and \"No new application code paths to audit.\"\n\n**6. After-count and coverage summary:**\n\n```bash\n# Count test files after generation\ngit ls-files 2>/dev/null | grep -E '(\\.test\\.|\\.spec\\.|_test\\.|_spec\\.)' | wc -l\n```\n\nFor PR body: `Tests: {before} \u2192 {after} (+{delta} new)`\nCoverage line: `Test Coverage Audit: N new code paths. M covered (X%). K tests generated, awaiting parent commit.`\n\n### Test Plan Artifact\n\nAfter producing the coverage diagram, write a test plan artifact so `/qa` and `/qa-only` can consume it:\n\n```bash\neval \"$(~/.claude/skills/gstack/bin/gstack-slug 2>/dev/null)\" && mkdir -p ~/.gstack/projects/$SLUG\nUSER=$(whoami)\nDATETIME=$(date +%Y%m%d-%H%M%S)\n```\n\nWrite to `~/.gstack/projects/{slug}/{user}-{branch}-ship-test-plan-{datetime}.md`:\n\n```markdown\n# Test Plan\nGenerated by /ship on {date}\nBranch: {branch}\nRepo: {owner/repo}\n\n## Affected Pages/Routes\n- {URL path} \u2014 {what to test and why}\n\n## Key Interactions to Verify\n- {interaction description} on {page}\n\n## Edge Cases\n- {edge case} on {page}\n\n## Critical Paths\n- {end-to-end flow that must work}\n```\n\nAfter your analysis, output a single JSON object on the LAST LINE of your response (no other text after it):\n{\"coverage_pct\":N,\"gaps\":N,\"diagram\":\"<full markdown coverage diagram for PR body>\",\"tests_added\":[\"path\",...]}\nUse null for an undetermined or skipped coverage percentage, not zero. Include every remaining gap in the diagram so the parent can target a second pass.\n````\n\n**Parent processing:**\n\n1. Read the subagent's final output. Parse the LAST line as JSON.\n2. Store `coverage_pct` (for Step 20 metrics), `gaps` (user summary), `tests_added` (for the commit).\n3. Embed `diagram` verbatim in the PR body's `## Test Coverage` section (Step 19).\n4. Print a one-line summary: `Coverage: {coverage_pct}%, {gaps} gaps. {tests_added.length} tests added.`\n\n**If the subagent fails, times out, returns invalid JSON, or never completes (backgrounded despite the flag, or no final output after ~10 minutes \u2014 stop waiting; if a backgrounded task is still running, stop it first so a late result never races the fallback):** Fall back to running the audit inline in the parent. Do not block /ship on subagent failure \u2014 partial results are better than none.\n\n\n**7. Coverage gate:**\n\nThe parent owns this gate after receiving the audit result, including after an inline fallback. Generated tests stay uncommitted until Step 15. Any further generation uses the same audit prompt with the remaining gaps and pass count supplied.\n\nBefore proceeding, check CLAUDE.md for a `## Test Coverage` section with `Minimum:` and `Target:` fields. If found, use those percentages. Otherwise use defaults: Minimum = 60%, Target = 80%.\n\nUsing the coverage percentage from the diagram in substep 4 (the `COVERAGE: X/Y (Z%)` line):\n\n- **>= target:** Pass. \"Coverage gate: PASS ({X}%).\" Continue.\n- **>= minimum, < target:** Use AskUserQuestion:\n - \"AI-assessed coverage is {X}%. {N} code paths are untested. Target is {target}%.\"\n - RECOMMENDATION: Choose A because untested code paths are where production bugs hide.\n - Options:\n A) Generate more tests for remaining gaps (recommended)\n B) Ship anyway \u2014 I accept the coverage risk\n C) These paths don't need tests \u2014 mark as intentionally uncovered\n - If A: Dispatch one more generation pass targeting remaining gaps, then re-evaluate the result here. Maximum 2 generation passes total. At the cap, offer only B/C or stop; do not offer another generation pass.\n - If B: Continue. Include in PR body: \"Coverage gate: {X}% \u2014 user accepted risk.\"\n - If C: Continue. Include in PR body: \"Coverage gate: {X}% \u2014 {N} paths intentionally uncovered.\"\n\n- **< minimum:** Use AskUserQuestion:\n - \"AI-assessed coverage is critically low ({X}%). {N} of {M} code paths have no tests. Minimum threshold is {minimum}%.\"\n - RECOMMENDATION: Choose A because less than {minimum}% means more code is untested than tested.\n - Options:\n A) Generate tests for remaining gaps (recommended)\n B) Override \u2014 ship with low coverage (I understand the risk)\n - If A: Dispatch one more generation pass. Maximum 2 passes total. At the cap, offer only B or stop; do not offer another generation pass.\n - If B: Continue. Include in PR body: \"Coverage gate: OVERRIDDEN at {X}%.\"\n\n**Coverage percentage undetermined:** If the coverage diagram doesn't produce a clear numeric percentage (ambiguous output, parse error), **skip the gate** with: \"Coverage gate: could not determine percentage \u2014 skipping.\" Do not default to 0% or block.\n\n**Test-only diffs:** Skip the gate (same as the existing fast-path).\n\n**100% coverage:** \"Coverage gate: PASS (100%).\" Continue.\n\n---"
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"id": "msg_011CetqpdSUxwZ1B7Fn4sVPq",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_01TBWEKwQAUmrA7KGb1ib5X4",
|
|
"name": "Agent",
|
|
"input": {
|
|
"description": "Test coverage audit of billing",
|
|
"subagent_type": "general-purpose",
|
|
"run_in_background": false,
|
|
"prompt": "You are running a ship-workflow test coverage audit. Working directory: /tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn. Branch feature/billing, base branch main. Do not commit or push. Perform only this audit; return unresolved user decisions to the parent instead of asking or advancing to another workflow step.\n\nSCOPE CONSTRAINTS FOR THIS RUN (from the user, override the default workflow):\n- DIAGRAM ONLY. Do NOT generate, write, or modify any test files. Do NOT write a test plan artifact to ~/.gstack. Do NOT run the test suite (it is mocked; assume tests pass).\n- `git diff main...HEAD` is empty on this branch (no commits beyond main), so treat the whole of src/billing.ts as the code under audit, and test/billing.test.ts as the existing tests.\n\n100% coverage is the goal \u2014 every untested path is a path where bugs hide. Evaluate what was ACTUALLY coded, not what was planned.\n\n### Test Framework Detection\n1. Read CLAUDE.md if present for a `## Testing` section. 2. Otherwise auto-detect:\n```bash\n[ -f package.json ] && echo \"RUNTIME:node\"\nls jest.config.* vitest.config.* playwright.config.* cypress.config.* 2>/dev/null\n[ -f package.json ] && grep -q '\"test\"[[:space:]]*:' package.json && echo \"SCRIPT:package.json test\"\ngit ls-files | grep -cE '\\.(test|spec)\\.[jt]sx?$' | sed 's/^/TESTFILES:/'\n```\n\n**0. Test file count (before):** `git ls-files | grep -E '(\\.test\\.|\\.spec\\.|_test\\.|_spec\\.)' | wc -l`\n\n**1. Trace every codepath** in src/billing.ts. Read the full file. For each exported function, follow data through every branch: every conditional/guard clause/early return, every error path (throw), every return. Consider edge inputs: 0, negative, NaN, empty string, wrong case currency, undefined.\n\n**2. Map user flows, interactions, and error states** for a payment/refund feature: e.g. complete purchase, invalid-amount error shown to user, unsupported-currency error, refund flow, refund missing reason. Mark payment/refund flows that warrant an integration test with [\u2192E2E] (auth/payment/data-destruction flows are too important to trust unit tests alone). No LLM calls exist, so no [\u2192EVAL].\n\n**3. Check each branch against existing tests** in test/billing.test.ts. Quality rubric:\n- \u2605\u2605\u2605 Tests behavior with edge cases AND error paths\n- \u2605\u2605 Tests correct behavior, happy path only\n- \u2605 Smoke test / trivial assertion\n\n**4. Output ASCII coverage diagram** in exactly this style, with BOTH code paths and user flows, marking [\u2192E2E] where appropriate, citing file:line for tested paths:\n\n```\nCODE PATHS USER FLOWS\n[+] src/billing.ts [+] Payment checkout\n \u251c\u2500\u2500 processPayment() \u251c\u2500\u2500 [\u2605\u2605 TESTED] ...\n \u2502 \u251c\u2500\u2500 [\u2605\u2605 TESTED] ... \u2014 billing.test.ts:7 \u251c\u2500\u2500 [GAP] [\u2192E2E] ...\n \u2502 \u251c\u2500\u2500 [GAP] ...\n \u2514\u2500\u2500 refundPayment() [+] Error states\n \u251c\u2500\u2500 [GAP] ... \u2514\u2500\u2500 [GAP] ...\n\nCOVERAGE: X/Y paths tested (Z%) | Code paths: a/b (p%) | User flows: c/d (q%)\nQUALITY: \u2605\u2605\u2605:n \u2605\u2605:n \u2605:n | GAPS: n (m E2E, 0 eval)\n```\nLegend: \u2605\u2605\u2605 behavior + edge + error | \u2605\u2605 happy path | \u2605 smoke check | [\u2192E2E] = needs integration test\n\n**5. Skip test generation** (user constraint). Note: \"Test generation skipped \u2014 diagram-only run requested.\"\n\n**6. Coverage summary line:** `Test Coverage Audit: N code paths. M covered (X%). 0 tests generated.` and `Tests: {before} \u2192 {before} (+0 new)`.\n\nAfter your analysis, output a single JSON object on the LAST LINE of your response (no other text after it):\n{\"coverage_pct\":N,\"gaps\":N,\"diagram\":\"<full markdown coverage diagram>\",\"tests_added\":[]}\nUse null for an undetermined coverage percentage, not zero. Include every gap in the diagram."
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": "toolu_01TBWEKwQAUmrA7KGb1ib5X4",
|
|
"message": {
|
|
"id": "msg_011CetqruyTK5WcUSDYLJ6bQ",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_016iWFR4FNySbDz65JT1LPG2",
|
|
"name": "Bash",
|
|
"input": {
|
|
"command": "ls -la && [ -f CLAUDE.md ] && grep -n -A10 '## Testing' CLAUDE.md; [ -f package.json ] && echo \"RUNTIME:node\"; ls jest.config.* vitest.config.* playwright.config.* cypress.config.* 2>/dev/null; [ -f package.json ] && grep -q '\"test\"[[:space:]]*:' package.json && echo \"SCRIPT:package.json test\"; git ls-files | grep -cE '\\.(test|spec)\\.[jt]sx?$' | sed 's/^/TESTFILES:/'; echo \"BEFORE: $(git ls-files | grep -E '(\\.test\\.|\\.spec\\.|_test\\.|_spec\\.)' | wc -l)\"; cat package.json 2>/dev/null; git diff main...HEAD --stat",
|
|
"description": "Detect test framework and count test files"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": "toolu_01TBWEKwQAUmrA7KGb1ib5X4",
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_016iWFR4FNySbDz65JT1LPG2",
|
|
"is_error": false,
|
|
"content": "total 16\ndrwx------ 7 vercel-sandbox vercel-sandbox 146 Sep 10 00:48 .\ndrwxr-xr-x 6 vercel-sandbox vercel-sandbox 112 Sep 10 00:51 ..\ndrwxr-xr-x 7 vercel-sandbox vercel-sandbox 150 Sep 10 00:48 .git\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 12 Sep 10 00:48 CHANGELOG.md\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 8 Sep 10 00:48 VERSION\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 182 Sep 10 00:48 package.json\ndrwxr-xr-x 4 vercel-sandbox vercel-sandbox 174 Sep 10 00:48 review\ndrwxr-xr-x 3 vercel-sandbox vercel-sandbox 59 Sep 10 00:48 ship\ndrwxr-xr-x 2 vercel-sandbox vercel-sandbox 24 Sep 10 00:48 src\ndrwxr-xr-x 2 vercel-sandbox vercel-sandbox 29 Sep 10 00:48 test\n-rw-r--r-- 1 vercel-sandbox vercel-sandbox 89 Sep 10 00:48 vitest.config.ts\nRUNTIME:node\nvitest.config.ts\nSCRIPT:package.json test\nTESTFILES:1\nBEFORE: 1\n{\n \"name\": \"test-coverage-app\",\n \"version\": \"1.0.0\",\n \"type\": \"module\",\n \"scripts\": {\n \"test\": \"echo \\\"no tests yet\\\"\"\n },\n \"devDependencies\": {\n \"vitest\": \"^1.0.0\"\n }\n}"
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": "toolu_01TBWEKwQAUmrA7KGb1ib5X4",
|
|
"message": {
|
|
"id": "msg_011CetqruyTK5WcUSDYLJ6bQ",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_01B2jdGhyiZ4WG6B2mkuj4D6",
|
|
"name": "Read",
|
|
"input": {
|
|
"file_path": "/tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn/src/billing.ts"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": "toolu_01TBWEKwQAUmrA7KGb1ib5X4",
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_01B2jdGhyiZ4WG6B2mkuj4D6",
|
|
"content": "1\t\n2\texport function processPayment(amount: number, currency: string) {\n3\t if (amount <= 0) throw new Error('Invalid amount');\n4\t if (currency !== 'USD' && currency !== 'EUR') throw new Error('Unsupported currency');\n5\t return { status: 'success', amount, currency };\n6\t}\n7\t\n8\texport function refundPayment(paymentId: string, reason: string) {\n9\t if (!paymentId) throw new Error('Payment ID required');\n10\t if (!reason) throw new Error('Reason required');\n11\t return { status: 'refunded', paymentId, reason };\n12\t}\n13\t"
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "assistant",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": "toolu_01TBWEKwQAUmrA7KGb1ib5X4",
|
|
"message": {
|
|
"id": "msg_011CetqruyTK5WcUSDYLJ6bQ",
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_01YNa4n5DK6JudmxVrDm3XQ4",
|
|
"name": "Read",
|
|
"input": {
|
|
"file_path": "/tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn/test/billing.test.ts"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": "toolu_01TBWEKwQAUmrA7KGb1ib5X4",
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_01YNa4n5DK6JudmxVrDm3XQ4",
|
|
"content": "1\t\n2\timport { describe, test, expect } from 'vitest';\n3\timport { processPayment } from '../src/billing';\n4\t\n5\tdescribe('processPayment', () => {\n6\t test('processes valid payment', () => {\n7\t const result = processPayment(100, 'USD');\n8\t expect(result.status).toBe('success');\n9\t });\n10\t // GAP: no test for invalid amount\n11\t // GAP: no test for unsupported currency\n12\t // GAP: refundPayment not tested at all\n13\t});\n14\t"
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"type": "user",
|
|
"session_id": "034c10a5-6b9c-49cd-8e3e-1759bce3d453",
|
|
"parent_tool_use_id": null,
|
|
"message": {
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_01TBWEKwQAUmrA7KGb1ib5X4",
|
|
"content": [
|
|
{
|
|
"type": "text",
|
|
"text": "Framework detected: Node runtime, **vitest** (`vitest.config.ts` present), `package.json` test script is a stub (`echo \"no tests yet\"`), 1 tracked test file. No CLAUDE.md. `git diff main...HEAD` is empty, so per instruction the whole of `src/billing.ts` is under audit against `test/billing.test.ts`.\n\n**Test file count (before):** 1\n\n## 1. Codepath trace \u2014 `/tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn/src/billing.ts`\n\n`processPayment(amount, currency)` (lines 2\u20136)\n- L3 guard `amount <= 0` \u2192 throws `'Invalid amount'` (catches 0 and negatives)\n- L4 guard `currency !== 'USD' && currency !== 'EUR'` \u2192 throws `'Unsupported currency'` (catches `'GBP'`, `''`, `undefined`, and lowercase `'usd'` \u2014 case-sensitive)\n- L5 success return `{status:'success', amount, currency}` \u2014 two accepting branches: USD and EUR\n- Edge finding: `NaN <= 0` and `undefined <= 0` are both `false`, so `processPayment(NaN, 'USD')` returns `status:'success'` with `amount: NaN`. No guard for that \u2014 a latent bug that an edge-case test would expose.\n\n`refundPayment(paymentId, reason)` (lines 8\u201312)\n- L9 guard `!paymentId` \u2192 throws `'Payment ID required'` (`''`, `undefined`)\n- L10 guard `!reason` \u2192 throws `'Reason required'`\n- L11 success return `{status:'refunded', paymentId, reason}`\n\n## 2\u20133. Existing tests \u2014 `/tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn/test/billing.test.ts`\n\nOne test (L6\u20139): `processPayment(100, 'USD')`, asserts only `result.status === 'success'`. Happy path, no assertion on the echoed `amount`/`currency` \u2192 \u2605\u2605. `refundPayment` is not imported at all. The file's own comments (L10\u201312) acknowledge the gaps.\n\n## 4. Coverage diagram\n\n```\nCODE PATHS USER FLOWS\n[+] src/billing.ts [+] Payment checkout\n \u251c\u2500\u2500 processPayment() \u251c\u2500\u2500 [\u2605\u2605 TESTED] Complete purchase (USD) \u2014 billing.test.ts:7\n \u2502 \u251c\u2500\u2500 [\u2605\u2605 TESTED] valid USD \u2192 {status:'success'} \u2014 billing.test.ts:7 \u251c\u2500\u2500 [GAP] Complete purchase (EUR)\n \u2502 \u251c\u2500\u2500 [GAP] valid EUR \u2192 success (L4 second accepting branch) \u2514\u2500\u2500 [GAP] [\u2192E2E] Checkout end-to-end (payment flow is unit-only today)\n \u2502 \u251c\u2500\u2500 [GAP] amount 0 \u2192 throws 'Invalid amount' (L3)\n \u2502 \u251c\u2500\u2500 [GAP] amount negative \u2192 throws 'Invalid amount' (L3) [+] Error states\n \u2502 \u251c\u2500\u2500 [GAP] amount NaN/undefined \u2192 passes guard, returns \u251c\u2500\u2500 [GAP] Invalid-amount error shown (0 / negative)\n \u2502 \u2502 success with amount:NaN (L3 latent bug) \u251c\u2500\u2500 [GAP] Unsupported-currency error (GBP, '', lowercase 'usd')\n \u2502 \u2514\u2500\u2500 [GAP] currency not USD/EUR ('GBP', '', 'usd') \u2192 \u2514\u2500\u2500 [GAP] NaN/undefined amount accepted silently (no error surfaced)\n \u2502 throws 'Unsupported currency' (L4)\n \u2514\u2500\u2500 refundPayment() [+] Refund\n \u251c\u2500\u2500 [GAP] empty/undefined paymentId \u2192 throws \u251c\u2500\u2500 [GAP] [\u2192E2E] Refund flow completes \u2192 status 'refunded'\n \u2502 'Payment ID required' (L9) \u251c\u2500\u2500 [GAP] Refund missing payment ID \u2192 error\n \u251c\u2500\u2500 [GAP] empty/undefined reason \u2192 throws \u2514\u2500\u2500 [GAP] Refund missing reason \u2192 error\n \u2502 'Reason required' (L10)\n \u2514\u2500\u2500 [GAP] valid \u2192 {status:'refunded', paymentId, reason} (L11)\n\nCOVERAGE: 2/17 paths tested (12%) | Code paths: 1/9 (11%) | User flows: 1/8 (13%)\nQUALITY: \u2605\u2605\u2605:0 \u2605\u2605:1 \u2605:0 | GAPS: 15 (2 E2E, 0 eval)\n```\nLegend: \u2605\u2605\u2605 behavior + edge + error | \u2605\u2605 happy path | \u2605 smoke check | [\u2192E2E] = needs integration test\n\n## 5. Test generation\n\nTest generation skipped \u2014 diagram-only run requested.\n\n## 6. Summary\n\nTest Coverage Audit: 9 code paths. 1 covered (11%). 0 tests generated.\nTests: 1 \u2192 1 (+0 new)\n\nKey findings for the parent:\n- Every error path (`Invalid amount`, `Unsupported currency`, `Payment ID required`, `Reason required`) is untested; `refundPayment` has zero coverage.\n- The one existing test only asserts `status`, not the echoed `amount`/`currency`.\n- Latent bug: `processPayment(NaN, 'USD')` / `processPayment(undefined as any, 'USD')` succeed because `NaN <= 0` is `false`. Whether to add a `Number.isFinite(amount)` guard is a user decision (out of scope for a diagram-only run).\n- `package.json` `test` script is a stub (`echo \"no tests yet\"`) even though vitest is configured \u2014 running `npm test` does not execute the suite. Whether to wire it to `vitest run` is a user decision.\n- Both the payment checkout and refund flows are payment-critical and warrant integration tests beyond unit coverage.\n\n{\"coverage_pct\":12,\"gaps\":15,\"diagram\":\"```\\nCODE PATHS USER FLOWS\\n[+] src/billing.ts [+] Payment checkout\\n \u251c\u2500\u2500 processPayment() \u251c\u2500\u2500 [\u2605\u2605 TESTED] Complete purchase (USD) \u2014 billing.test.ts:7\\n \u2502 \u251c\u2500\u2500 [\u2605\u2605 TESTED] valid USD \u2192 {status:'success'} \u2014 billing.test.ts:7 \u251c\u2500\u2500 [GAP] Complete purchase (EUR)\\n \u2502 \u251c\u2500\u2500 [GAP] valid EUR \u2192 success (L4 second accepting branch) \u2514\u2500\u2500 [GAP] [\u2192E2E] Checkout end-to-end (payment flow is unit-only today)\\n \u2502 \u251c\u2500\u2500 [GAP] amount 0 \u2192 throws 'Invalid amount' (L3)\\n \u2502 \u251c\u2500\u2500 [GAP] amount negative \u2192 throws 'Invalid amount' (L3) [+] Error states\\n \u2502 \u251c\u2500\u2500 [GAP] amount NaN/undefined \u2192 passes guard, returns \u251c\u2500\u2500 [GAP] Invalid-amount error shown (0 / negative)\\n \u2502 \u2502 success with amount:NaN (L3 latent bug) \u251c\u2500\u2500 [GAP] Unsupported-currency error (GBP, '', lowercase 'usd')\\n \u2502 \u2514\u2500\u2500 [GAP] currency not USD/EUR ('GBP', '', 'usd') \u2192 \u2514\u2500\u2500 [GAP] NaN/undefined amount accepted silently (no error surfaced)\\n \u2502 throws 'Unsupported currency' (L4)\\n \u2514\u2500\u2500 refundPayment() [+] Refund\\n \u251c\u2500\u2500 [GAP] empty/undefined paymentId \u2192 throws \u251c\u2500\u2500 [GAP] [\u2192E2E] Refund flow completes \u2192 status 'refunded'\\n \u2502 'Payment ID required' (L9) \u251c\u2500\u2500 [GAP] Refund missing payment ID \u2192 error\\n \u251c\u2500\u2500 [GAP] empty/undefined reason \u2192 throws \u2514\u2500\u2500 [GAP] Refund missing reason \u2192 error\\n \u2502 'Reason required' (L10)\\n \u2514\u2500\u2500 [GAP] valid \u2192 {status:'refunded', paymentId, reason} (L11)\\n\\nCOVERAGE: 2/17 paths tested (12%) | Code paths: 1/9 (11%) | User flows: 1/8 (13%)\\nQUALITY: \u2605\u2605\u2605:0 \u2605\u2605:1 \u2605:0 | GAPS: 15 (2 E2E, 0 eval)\\n```\\nLegend: \u2605\u2605\u2605 behavior + edge + error | \u2605\u2605 happy path | \u2605 smoke check | [\u2192E2E] = needs integration test\",\"tests_added\":[]}"
|
|
},
|
|
{
|
|
"type": "text",
|
|
"text": "agentId: aa60f3912ccfae784 (use SendMessage with to: 'aa60f3912ccfae784', summary: '<5-10 word recap>' to continue this agent)\n<usage>subagent_tokens: 19229\ntool_uses: 3\nduration_ms: 62884</usage>"
|
|
}
|
|
]
|
|
}
|
|
]
|
|
}
|
|
}
|
|
],
|
|
"toolCalls": [
|
|
{
|
|
"tool": "Bash",
|
|
"input": {
|
|
"command": "ls -la && find . -name \"SKILL.md\" -not -path \"./node_modules/*\" 2>/dev/null",
|
|
"description": "List project and locate SKILL.md"
|
|
}
|
|
},
|
|
{
|
|
"tool": "Bash",
|
|
"input": {
|
|
"command": "cat ship/SKILL.md",
|
|
"description": "Read ship skill instructions"
|
|
}
|
|
},
|
|
{
|
|
"tool": "Bash",
|
|
"input": {
|
|
"command": "cat -n src/billing.ts",
|
|
"description": "Read billing source"
|
|
}
|
|
},
|
|
{
|
|
"tool": "Bash",
|
|
"input": {
|
|
"command": "cat -n test/billing.test.ts",
|
|
"description": "Read billing tests"
|
|
}
|
|
},
|
|
{
|
|
"tool": "Bash",
|
|
"input": {
|
|
"command": "git log --oneline main..HEAD; echo \"---diff vs main---\"; git diff main --stat; echo \"---\"; cat package.json",
|
|
"description": "Show branch diff vs main and package.json"
|
|
}
|
|
},
|
|
{
|
|
"tool": "Bash",
|
|
"input": {
|
|
"command": "grep -n -i \"3\\.4\\|coverage audit\\|Test Coverage\" ship/SKILL.md | head -30",
|
|
"description": "Locate Step 3.4 in SKILL.md"
|
|
}
|
|
},
|
|
{
|
|
"tool": "Bash",
|
|
"input": {
|
|
"command": "sed -n 515,560p ship/SKILL.md; echo \"=== 680-720 ===\"; sed -n 680,720p ship/SKILL.md; echo \"=== sections dir ===\"; ls -la ship/sections 2>/dev/null; ls ~/.claude/skills/gstack/ship/sections/ 2>/dev/null",
|
|
"description": "Read coverage-audit step and locate sections file"
|
|
}
|
|
},
|
|
{
|
|
"tool": "Bash",
|
|
"input": {
|
|
"command": "cat ship/sections/test-coverage.md 2>/dev/null || cat ~/.claude/skills/gstack/ship/sections/test-coverage.md 2>/dev/null || echo \"NOT FOUND\"",
|
|
"description": "Read test-coverage section"
|
|
}
|
|
},
|
|
{
|
|
"tool": "Agent",
|
|
"input": {
|
|
"description": "Test coverage audit of billing",
|
|
"subagent_type": "general-purpose",
|
|
"run_in_background": false,
|
|
"prompt": "You are running a ship-workflow test coverage audit. Working directory: /tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn. Branch feature/billing, base branch main. Do not commit or push. Perform only this audit; return unresolved user decisions to the parent instead of asking or advancing to another workflow step.\n\nSCOPE CONSTRAINTS FOR THIS RUN (from the user, override the default workflow):\n- DIAGRAM ONLY. Do NOT generate, write, or modify any test files. Do NOT write a test plan artifact to ~/.gstack. Do NOT run the test suite (it is mocked; assume tests pass).\n- `git diff main...HEAD` is empty on this branch (no commits beyond main), so treat the whole of src/billing.ts as the code under audit, and test/billing.test.ts as the existing tests.\n\n100% coverage is the goal \u2014 every untested path is a path where bugs hide. Evaluate what was ACTUALLY coded, not what was planned.\n\n### Test Framework Detection\n1. Read CLAUDE.md if present for a `## Testing` section. 2. Otherwise auto-detect:\n```bash\n[ -f package.json ] && echo \"RUNTIME:node\"\nls jest.config.* vitest.config.* playwright.config.* cypress.config.* 2>/dev/null\n[ -f package.json ] && grep -q '\"test\"[[:space:]]*:' package.json && echo \"SCRIPT:package.json test\"\ngit ls-files | grep -cE '\\.(test|spec)\\.[jt]sx?$' | sed 's/^/TESTFILES:/'\n```\n\n**0. Test file count (before):** `git ls-files | grep -E '(\\.test\\.|\\.spec\\.|_test\\.|_spec\\.)' | wc -l`\n\n**1. Trace every codepath** in src/billing.ts. Read the full file. For each exported function, follow data through every branch: every conditional/guard clause/early return, every error path (throw), every return. Consider edge inputs: 0, negative, NaN, empty string, wrong case currency, undefined.\n\n**2. Map user flows, interactions, and error states** for a payment/refund feature: e.g. complete purchase, invalid-amount error shown to user, unsupported-currency error, refund flow, refund missing reason. Mark payment/refund flows that warrant an integration test with [\u2192E2E] (auth/payment/data-destruction flows are too important to trust unit tests alone). No LLM calls exist, so no [\u2192EVAL].\n\n**3. Check each branch against existing tests** in test/billing.test.ts. Quality rubric:\n- \u2605\u2605\u2605 Tests behavior with edge cases AND error paths\n- \u2605\u2605 Tests correct behavior, happy path only\n- \u2605 Smoke test / trivial assertion\n\n**4. Output ASCII coverage diagram** in exactly this style, with BOTH code paths and user flows, marking [\u2192E2E] where appropriate, citing file:line for tested paths:\n\n```\nCODE PATHS USER FLOWS\n[+] src/billing.ts [+] Payment checkout\n \u251c\u2500\u2500 processPayment() \u251c\u2500\u2500 [\u2605\u2605 TESTED] ...\n \u2502 \u251c\u2500\u2500 [\u2605\u2605 TESTED] ... \u2014 billing.test.ts:7 \u251c\u2500\u2500 [GAP] [\u2192E2E] ...\n \u2502 \u251c\u2500\u2500 [GAP] ...\n \u2514\u2500\u2500 refundPayment() [+] Error states\n \u251c\u2500\u2500 [GAP] ... \u2514\u2500\u2500 [GAP] ...\n\nCOVERAGE: X/Y paths tested (Z%) | Code paths: a/b (p%) | User flows: c/d (q%)\nQUALITY: \u2605\u2605\u2605:n \u2605\u2605:n \u2605:n | GAPS: n (m E2E, 0 eval)\n```\nLegend: \u2605\u2605\u2605 behavior + edge + error | \u2605\u2605 happy path | \u2605 smoke check | [\u2192E2E] = needs integration test\n\n**5. Skip test generation** (user constraint). Note: \"Test generation skipped \u2014 diagram-only run requested.\"\n\n**6. Coverage summary line:** `Test Coverage Audit: N code paths. M covered (X%). 0 tests generated.` and `Tests: {before} \u2192 {before} (+0 new)`.\n\nAfter your analysis, output a single JSON object on the LAST LINE of your response (no other text after it):\n{\"coverage_pct\":N,\"gaps\":N,\"diagram\":\"<full markdown coverage diagram>\",\"tests_added\":[]}\nUse null for an undetermined coverage percentage, not zero. Include every gap in the diagram."
|
|
}
|
|
},
|
|
{
|
|
"tool": "Bash",
|
|
"input": {
|
|
"command": "ls -la && [ -f CLAUDE.md ] && grep -n -A10 '## Testing' CLAUDE.md; [ -f package.json ] && echo \"RUNTIME:node\"; ls jest.config.* vitest.config.* playwright.config.* cypress.config.* 2>/dev/null; [ -f package.json ] && grep -q '\"test\"[[:space:]]*:' package.json && echo \"SCRIPT:package.json test\"; git ls-files | grep -cE '\\.(test|spec)\\.[jt]sx?$' | sed 's/^/TESTFILES:/'; echo \"BEFORE: $(git ls-files | grep -E '(\\.test\\.|\\.spec\\.|_test\\.|_spec\\.)' | wc -l)\"; cat package.json 2>/dev/null; git diff main...HEAD --stat",
|
|
"description": "Detect test framework and count test files"
|
|
}
|
|
},
|
|
{
|
|
"tool": "Read",
|
|
"input": {
|
|
"file_path": "/tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn/src/billing.ts"
|
|
}
|
|
},
|
|
{
|
|
"tool": "Read",
|
|
"input": {
|
|
"file_path": "/tmp/gstack-paid-shard-6fJnEY/tmp/skill-e2e-coverage-wQ70Nn/test/billing.test.ts"
|
|
}
|
|
}
|
|
],
|
|
"durationMs": 120026,
|
|
"costUsd": 0,
|
|
"turnsUsed": 0,
|
|
"model": "claude-fable-5-1"
|
|
}
|
|
}
|
|
]
|
|
}
|