gstack/test/fixtures/coverage-audit-sed-context.json

{
 "provenance": {
  "source": "CI run 37087797888 attempt 1, eval slice 12, native capture plan-eng-coverage-audit.ndjson (2026-10-03)",
  "recordedPassed": false,
  "recordedFailures": [
   "missing successful source-file read",
   "missing successful test-file read"
  ],
  "schema": "Exact public init cwd/session, the one Bash call that read both owned files, its complete successful tool result, and the final diagram text."
 },
 "cwd": "/home/runner/.cache/gstack-paid-shard-p75JPs/tmp/skill-e2e-plan-eng-review-coverage-cJMx3z",
 "sessionId": "7851a911-2ba2-4ae7-b10a-8180a20300ba",
 "toolUseId": "toolu_01JnHVoaTcBgLFP1imrkJGSd",
 "command": "sed -n 542,560p plan-eng-review/sections/review-sections.md && echo ... && sed -n 650,735p plan-eng-review/sections/review-sections.md && echo ===== && sed -n 1118,1127p plan-eng-review/sections/review-sections.md && echo ===== SRC && cat -n src/billing.ts && echo ===== TEST && cat -n test/billing.test.ts && echo ===== DIFF && git diff --stat main",
 "isError": false,
 "output": "### 3. Test review\n\nFor shared-code changes, audit existing/missing shared-contract tests (behavior,\nerrors, side effects, boundaries) and each migrated caller's integration/differences.\nRejected extractions still need coverage for real duplicated-code defects.\n\nCoverage goal: every changed behavior is protected by a test that would catch a real regression. Test count is not a goal. Identify the tests each planned codepath needs. Add required proof for an exact approved behavior without asking again; take new policies or optional verification depth through the decision gate before treating their tests as accepted work. Review the requirements here; do not build the proposed tests.\n\n#### Test Framework Detection\n\nBefore analyzing coverage, detect the project's test framework:\n\n1. **Read CLAUDE.md** — look for a `## Testing` section with test command and framework name. If found, use that as the authoritative source.\n2. **If CLAUDE.md has no testing section, auto-detect:**\n\n```bash\nsetopt +o nomatch 2>/dev/null || true  # zsh compat\n# Detect project runtime (markers are evidence, not commands to run blind)\n[ -f manage.py ] && echo \"RUNTIME:python FRAMEWORK:django\"\n...\nExample: Value: protects=refundPayment rejects an empty reason; fails_when=the reason guard is removed or inverted; why_new=billing.test.ts covers processPayment only; seam=none\nRejected (covered_elsewhere): \"checkout renders\"; checkout.e2e.ts:15 covers it, so extend that test.\n\nWeak tests (★ smoke/existence/trivial, gate-failing or unrated) never count as coverage. X = paths with a ★★/★★★ test / total paths (value-weighted; the gate uses X); Y = paths with any test / total paths. /ship computes them; here every proposed test needs a card.\n\nRetention bar: keep a test that independently enforces a public API, protocol, config, migration, storage, security, platform, default, prompt-byte, generated-output (golden), package, release or architecture contract; static or slow is no reason to delete.\n\n#### E2E Test Decision Matrix\n\nWhen checking each branch, also determine whether a unit test or E2E/integration test is the right tool:\n\n**RECOMMEND E2E (mark as [→E2E] in the diagram):**\n- Common user flow spanning 3+ components/services (e.g., signup → verify email → first login)\n- Integration point where mocking hides real failures (e.g., API → queue → worker → DB)\n- Auth/payment/data-destruction flows — too important to trust unit tests alone\n\n**RECOMMEND EVAL (mark as [→EVAL] in the diagram):**\n- Critical LLM call that needs a quality eval (e.g., prompt change → test output still meets quality bar)\n- Changes to prompt templates, system instructions, or tool definitions\n\n**STICK WITH UNIT TESTS:**\n- Pure function with clear inputs/outputs\n- Internal helper with no side effects\n- Edge case of a single function (null input, empty array)\n- Obscure/rare flow that isn't customer-facing\n\n#### REGRESSION RULE (mandatory)\n\n**IRON RULE:** When a planned change puts existing behavior at risk without regression coverage, that coverage is a critical requirement. Carry forward an exact approved regression contract; otherwise use one dedicated AskUserQuestion to settle it — behavior to preserve, intentional changes, and acceptance assertions — before adding the approved contract to the plan. Ask how to cover it, not whether to skip it. Do not silently include it under a different test-depth question.\n\nA proposed rewrite is a regression risk, not proof that running code already broke. Name the existing callers and behavior at risk; preserve unchanged behavior and explicitly identify intended differences. No skipping regression coverage.\n\n**Step 4. Output ASCII coverage diagram:**\n\nFor targeted audits, start Test review output with the coverage diagram. In full\nplan reviews, put it inside the normal Test review section. Required outputs\nkeep the final terminal report order.\n\nInclude BOTH code paths and user flows in the same diagram. Mark E2E-worthy and eval-worthy paths:\n\n```\nCODE PATHS                                            USER FLOWS\n[+] src/services/billing.ts                           [+] Payment checkout\n  ├── processPayment()                                  ├── [★★★ TESTED] Complete purchase — checkout.e2e.ts:15\n  │   ├── [★★★ TESTED] happy + declined + timeout      ├── [GAP] [→E2E] Double-click submit\n  │   ├── [GAP]         Network timeout                 └── [GAP]        Navigate away mid-payment\n  │   └── [GAP]         Invalid currency\n  └── refundPayment()                                 [+] Error states\n      ├── [★★  TESTED] Full refund — :89                ├── [★★  TESTED] Card declined message\n      └── [★   TESTED] Partial (non-throw only) — :101  └── [GAP]        Network timeout UX\n\nLLM integration: [GAP] [→EVAL] Prompt template change — needs eval test\n\nCOVERAGE: 5/13 paths tested (38%)  |  Code paths: 3/5 (60%)  |  User flows: 2/8 (25%)\nQUALITY: ★★★:2 ★★:2 ★:1  |  GAPS: 8 (2 E2E, 1 eval)\n```\n\nLegend: ★★★ behavior + edge + error  |  ★★ happy path  |  ★ smoke check\n[→E2E] = needs integration test  |  [→EVAL] = needs LLM eval\n\nAvoid bare `[ ]` or `[x]` in diagrams unless the block includes\n`Legend: [x] tested | [ ] no test`. Prefer `[GAP]`, `[★★ TESTED]`,\n`[→E2E]`, `[→EVAL]`; keep user-flow markers off code-path rows.\n\n**Fast path:** All paths covered → \"Test review: All new code paths have test coverage ✓\" Still check LLM/eval scope and produce the Test Plan Artifact below.\n\n#### LLM/eval scope\n\nFor LLM/prompt changes: check the \"Prompt/LLM changes\" file patterns listed in CLAUDE.md. If this plan touches ANY of those patterns, state which eval suites must be run, which cases should be added, and what baselines to compare against. Include unapproved eval scope among the choices resolved in Step 5.\n\n**Step 5. Add missing tests to the plan:**\n\nCollect the requirements for each GAP and the LLM/eval scope above. Carry forward required proof of approved behavior. Mark new contracts and optional depth choices pending until the decision gate below resolves them. For every proposed test, specify:\n- What test file to create (match existing naming conventions)\n- What the test should assert (specific inputs → expected outputs/behavior)\n- Whether it's a unit test, E2E test, or eval (use the decision matrix)\n- Its value card (test value bar above)\n- For regression risks: flag as **CRITICAL** and name the behavior to protect\n\nA proposal that fails the value bar becomes \"extend <existing test>\" or is dropped with a one-line reason. Also list **Tests made obsolete by this plan** (proposal only; retiring one still needs a complete retirement card at implementation time, see /test-audit).\n\nRun the decision gate for this section's new or reopened choices. **STOP for each pending decision.** Wait for its answer before applying that remedy, moving to the next section or calling ExitPlanMode.\n\nWhen these test and eval choices are resolved, write the Test Plan Artifact below. Its approved requirements should be specific enough to implement alongside the feature code.\n\n#### Test Plan Artifact\n=====\n### Diagrams\nDiagram non-trivial flows, states and pipelines in ASCII. Name files needing inline\ndiagrams for complex model, service or mixin behavior.\n\n### Failure modes\nFor each new diagrammed path, name a realistic production failure, its test/error\nhandling coverage, and whether users see a clear error or a silent failure.\n\nIf any failure mode has no test AND no error handling AND would be silent, flag it as a **critical gap**.\n\n===== SRC\n     1\t\n     2\texport function processPayment(amount: number, currency: string) {\n     3\t  if (amount <= 0) throw new Error('Invalid amount');\n     4\t  if (currency !== 'USD' && currency !== 'EUR') throw new Error('Unsupported currency');\n     5\t  return { status: 'success', amount, currency };\n     6\t}\n     7\t\n     8\texport function refundPayment(paymentId: string, reason: string) {\n     9\t  if (!paymentId) throw new Error('Payment ID required');\n    10\t  if (!reason) throw new Error('Reason required');\n    11\t  return { status: 'refunded', paymentId, reason };\n    12\t}\n    13\t\n    14\t// coverage-read-evidence: e5173c10-71e8-4397-abbb-4e35dcffb249\n===== TEST\n     1\t\n     2\timport { describe, test, expect } from 'vitest';\n     3\timport { processPayment } from '../src/billing';\n     4\t\n     5\tdescribe('processPayment', () => {\n     6\t  test('processes valid payment', () => {\n     7\t    const result = processPayment(100, 'USD');\n     8\t    expect(result.status).toBe('success');\n     9\t  });\n    10\t  // GAP: no test for invalid amount\n    11\t  // GAP: no test for unsupported currency\n    12\t  // GAP: refundPayment not tested at all\n    13\t});\n    14\t\n    15\t// coverage-read-evidence: 38fbe325-18be-46ee-ba75-7f3d17c581a2\n===== DIFF\n src/billing.ts       | 2 ++\n test/billing.test.ts | 2 ++\n 2 files changed, 4 insertions(+)",
 "finalOutput": "## Test review (section 3 only) — `src/billing.ts` on `feature/billing` vs `main`\n\n**Framework:** vitest (detected from `vitest.config.ts`, `package.json`, and the test import). No CLAUDE.md testing section exists.\n\n**Codepath trace.** Both functions are pure, side-effect-free validators, so unit tests are the right tool throughout. No E2E or eval paths apply.\n\n```\nCODE PATHS                                                      USER FLOWS\n[+] src/billing.ts                                              [+] Payment checkout (pure fn, no UI/IO in scope)\n  ├── processPayment(amount, currency)                            ├── [★★  TESTED] Valid USD payment — billing.test.ts:6\n  │   ├── [★★  TESTED] happy path: 100 USD → status 'success' — :6  └── [GAP]        Valid EUR payment (second accepted branch)\n  │   │                 (asserts status only; amount/currency\n  │   │                  echo-back unasserted)                  [+] Payment rejection\n  │   ├── [GAP]         amount <= 0 → 'Invalid amount'            ├── [GAP]        Zero / negative amount rejected\n  │   │                 (boundary 0, negative)                    └── [GAP]        Unsupported currency rejected (e.g. 'GBP')\n  │   └── [GAP]         currency ∉ {USD, EUR} → 'Unsupported currency'\n  │                     (EUR true-branch also untested)         [+] Refund\n  └── refundPayment(paymentId, reason)                            ├── [GAP]        Successful refund\n      ├── [GAP]         happy path → status 'refunded'            ├── [GAP]        Missing payment ID rejected\n      ├── [GAP]         !paymentId → 'Payment ID required'        └── [GAP]        Missing reason rejected\n      └── [GAP]         !reason → 'Reason required'\n\nLLM integration: none in scope\n\nCOVERAGE: 2/12 paths tested (17%)  |  Code paths: 1/6 (17%)  |  User flows: 1/6 (17%)\nQUALITY: ★★★:0 ★★:1 ★:0  |  GAPS: 10 (0 E2E, 0 eval)\n```\n\nLegend: ★★★ behavior + edge + error  |  ★★ happy path  |  ★ smoke check\n[→E2E] = needs integration test  |  [→EVAL] = needs LLM eval\n\n**Coverage summary**\n\n- **processPayment** has one ★★ happy-path test. Both error guards are untested, so removing or inverting either `throw` would pass the suite. The EUR branch of the currency check is also never exercised.\n- **refundPayment** has zero tests. It is not even imported in the test file. All three of its paths are gaps.\n- **Value-weighted X** is 1/6 code paths with a ★★ or better test. The one existing test is one assertion short of ★★★ because it checks only the status field, not the echoed amount and currency.\n- **Regression risk:** both functions are exported public API. Any later refactor of the guards has no safety net today. This is a regression-coverage gap under the iron rule, but no plan change is being proposed in this audit, so no decision gate is raised here.\n\n**Required tests to close the gaps** (requirements only, not built, per your scope):\n\n| Target | Test to add in `test/billing.test.ts` | Type | Value card |\n|---|---|---|---|\n| processPayment | `processPayment(0,'USD')` and `(-1,'USD')` throw `'Invalid amount'` | unit | protects=amount guard incl. boundary 0; fails_when=guard removed or `<=` becomes `<`; why_new=only happy path tested; seam=none |\n| processPayment | `processPayment(100,'GBP')` throws `'Unsupported currency'`; `(100,'EUR')` succeeds | unit | protects=currency allow-list both branches; fails_when=allow-list widened or EUR dropped; why_new=EUR and reject branch untested; seam=none |\n| processPayment | extend `:6` to assert `amount` and `currency` echo back | extend | protects=return shape contract; fails_when=fields dropped or swapped; why_new=existing test asserts status only; seam=none |\n| refundPayment | `refundPayment('p1','dup')` returns `{status:'refunded', paymentId, reason}` | unit | protects=refund return contract; fails_when=status string or echo changes; why_new=function has no tests; seam=none |\n| refundPayment | `refundPayment('','x')` throws `'Payment ID required'` | unit | protects=paymentId guard; fails_when=guard removed; why_new=function has no tests; seam=none |\n| refundPayment | `refundPayment('p1','')` throws `'Reason required'` | unit | protects=reason guard; fails_when=guard removed or inverted; why_new=billing.test.ts covers processPayment only; seam=none |\n\nSource and tests were not modified. Architecture, code quality, performance, outside voice, and test generation were skipped as requested.",
 "files": {
  "source": "\nexport function processPayment(amount: number, currency: string) {\n  if (amount <= 0) throw new Error('Invalid amount');\n  if (currency !== 'USD' && currency !== 'EUR') throw new Error('Unsupported currency');\n  return { status: 'success', amount, currency };\n}\n\nexport function refundPayment(paymentId: string, reason: string) {\n  if (!paymentId) throw new Error('Payment ID required');\n  if (!reason) throw new Error('Reason required');\n  return { status: 'refunded', paymentId, reason };\n}\n\n// coverage-read-evidence: e5173c10-71e8-4397-abbb-4e35dcffb249\n",
  "tests": "\nimport { describe, test, expect } from 'vitest';\nimport { processPayment } from '../src/billing';\n\ndescribe('processPayment', () => {\n  test('processes valid payment', () => {\n    const result = processPayment(100, 'USD');\n    expect(result.status).toBe('success');\n  });\n  // GAP: no test for invalid amount\n  // GAP: no test for unsupported currency\n  // GAP: refundPayment not tested at all\n});\n\n// coverage-read-evidence: 38fbe325-18be-46ee-ba75-7f3d17c581a2\n"
 }
}