gstack/test/fixtures/coverage-audit-two-file-grep.json

{
  "provenance": {
    "source": "https://github.com/garrytan/gstack/actions/runs/41149166998 eval slice 5 (train gate, tested cab8ef12), review-coverage-audit (claude-opus-5-5); the second parent Bash call and its result, byte-exact",
    "recordedPassed": false,
    "recordedFailures": [
      "missing successful source-file read",
      "missing successful test-file read"
    ],
    "note": "The first call printed 34 KB and the native result kept only a 2 KB preview, so it delivered neither file. The second call read both files in a ; chain that ended in a grep display with an escaped alternation over two literal in-cwd files; the escaped-grep grammar accepted one file operand only, so the whole chain got no credit."
  },
  "cwd": "/home/runner/.cache/gstack-paid-shard-O49Xex/tmp/skill-e2e-review-coverage-AWwx64",
  "sessionId": "2301d71e-8faf-4efd-8137-56549a57bf86",
  "call": {
    "toolUseId": "toolu_01NRrh5xY6cAXURo6ZJ6Ucoy",
    "command": "cd /home/runner/.cache/gstack-paid-shard-O49Xex/tmp/skill-e2e-review-coverage-AWwx64; cat review/specialists/testing.md; echo ----; cat -n src/billing.ts; echo ----; cat -n test/billing.test.ts; echo ----; grep -n -i -A3 \"coverage diagram\\|ASCII\" review/SKILL.md review/sections/review-army.md | head -80",
    "isError": false,
    "output": "# Testing Specialist Review Checklist\n\nScope: Always-on (every review)\nOutput: JSON objects, one finding per line. Schema:\n{\"severity\":\"CRITICAL|INFORMATIONAL\",\"confidence\":N,\"path\":\"file\",\"line\":N,\"category\":\"testing\",\"summary\":\"...\",\"fix\":\"...\",\"fingerprint\":\"path:line:testing\",\"specialist\":\"testing\"}\nOptional: line, fix, fingerprint, evidence, test_stub.\nIf no findings: output `NO FINDINGS` and nothing else.\n\nIf the caller explicitly asks for an ASCII coverage diagram, first read the\nnamed source and test files directly in a dedicated tool call. Keep that tool\ncall limited to those two file reads, using either native Read calls or a simple\nshell display such as `cat -n src/file && echo ---- && cat -n test/file`. Read\ndiffs, package files, configs, or other context in separate tool calls. Then\noutput the diagram before any JSON findings. Use one function root per public or\nchanged function, put the `[OK]` or `[GAP]` marker on the same branch row as the\ntested or missing path, and keep the legend inside the same diagram block:\n\n```text\nsrc/billing.ts\nprocessPayment(amount, currency)\n├── valid USD happy path returns success [OK]\n└── invalid amount / unsupported currency branches [GAP]\nrefundPayment(paymentId, reason)\n└── refund success and guard branches not imported or untested [GAP]\nLegend: [OK] tested [GAP] no test\n```\n\nDo not rely on a summary table, prose paragraph, or distant nested marker as the\nonly coverage evidence. Covered happy-path rows must name the successful, valid,\nor concrete tested input path; gap rows must stay under the function that owns\nthe missing path.\n\n---\n\n## Categories\n\n### Exploratory hypotheses\n\nThe parent runs one shared exploratory QA pass before Fix-First, including small diffs\nthat skip specialists. Supply high-risk changed contracts, adverse scenarios and native\ntest candidates to that pass; do not launch another explorer, edit product/tests or\ncommit. A test proposal uses `test_stub` and keeps the caller's approval requirements.\nCheck real request/queue/storage effects, retries, duplicates and interrupted recovery\nwhere applicable. A diagram or test count is not executed proof. For discoveries promoted\nto regressions, require failure for the reproduced bug before repair and green plus\noriginal/adjacent probes afterward; never accept buggy-output goldens or discarded valid\nred tests. Unit tests suit logic; real integration/E2E tests protect boundaries mocks hide.\n\n### Missing Negative-Path Tests\n- New code paths that handle errors, rejections, or invalid input with NO corresponding test\n- Guard clauses and early returns that are untested\n- Error branches in try/catch, rescue, or error boundaries with no failure-path test\n- Permission/auth checks that are asserted in code but never tested for the \"denied\" case\n\n### Missing Edge-Case Coverage\n- Boundary values: zero, negative, max-int, empty string, empty array, nil/null/undefined\n- Single-element collections (off-by-one on loops)\n- Unicode and special characters in user-facing inputs\n- Concurrent access patterns with no race-condition test\n\n### Negative Assertions\n- A new assertion that something is absent (`not.toContain`, `not in`, `assertNotIn`,\n  `pytest.raises` on a call that used to succeed) records a behavior removal, not coverage.\n  Cite the spec line, plan item or PR text that authorizes the removal; with none, report it\n  as a product decision shipped as a test.\n- A rename or removal in this diff can empty a standing negative assertion: a test asserting\n  the old token is absent now passes forever and guards nothing. For each identifier, flag,\n  heading or string the diff renames or deletes, grep the test suite for negative assertions\n  naming the old value, including inside tests that assert something else, and report each\n  as a coverage gap whose fix names the new value.\n\n### Test Isolation Violations\n- Tests sharing mutable state (class variables, global singletons, DB records not cleaned up)\n- Order-dependent tests (pass in sequence, fail when randomized)\n- Tests that depend on system clock, timezone, or locale\n- Tests that make real network calls instead of using stubs/mocks\n\n### Flaky Test Patterns\n- Timing-dependent assertions (sleep, setTimeout, waitFor with tight timeouts)\n- Assertions on ordering of unordered results (hash keys, Set iteration, async resolution order)\n- Tests that depend on external services (APIs, databases) without fallback\n- Randomized test data without seed control\n\n### Security Enforcement Tests Missing\n- Auth/authz checks in controllers with no test for the \"unauthorized\" case\n- Rate limiting logic with no test proving it actually blocks\n- Input sanitization with no test for malicious input\n- CSRF/CORS configuration with no integration test\n\n### Coverage Gaps\n- New public methods/functions with zero test coverage\n- Changed methods where existing tests only cover the old behavior, not the new branch\n- Utility functions called from multiple places but tested only indirectly\n- A path whose only tests are weak (★ smoke/existence/trivial, or a new test failing the\n  authoring gate below) stays a coverage gap at its existing severity; a low-value test\n  never closes a gap\n\n### Low-value or implementation-coupled tests\nScope: tests and test-only production seams added or changed in the diff. Findings are\nINFORMATIONAL, never CRITICAL, and never an auto-delete: recommend rewrite at the owning\nboundary, extend an existing test, or retire with a complete retirement card\n(`test, detects, non_test_callers, search_command, stronger_proof, history, unlocks,\nvalidation`). End each finding's `fix` with `Repo-wide sweep: run /test-audit.`\n\nA new or changed test passes the authoring gate only when all four answers exist (read\nits `Value: protects=...; fails_when=...; why_new=...; seam=...` header comment when the\ndiff has one; an optional `no_claim=...` names what green does not prove and which\nfixtures or mocks stand in for what, and never waives a required live check):\n1. What observable behavior, invariant or independent contract does it protect?\n2. What credible regression makes it fail?\n3. Why does existing coverage not already catch that? Prefer adding a row to an existing table-driven test or shared fixture over a near-duplicate.\n4. Does it need a production seam (export, flag, wrapper, injection hook) that no production caller needs? If yes, test at the real boundary instead.\n\nPatterns:\n- assertion-free coverage probes\n- self-comparisons and identity copies\n- copied fixtures, inventories or export lists\n- exact source, import or string greps that are not a declared contract\n- private predicate or call-shape tests duplicated at a real boundary\n- duplicate invocations of the same contract\n- per-caller replays of a shared helper's tests\n- tests whose only purpose is keeping a test-only export, global or wrapper alive\n- production code whose only callers are tests\n\nRetention bar (never flag): a test that independently enforces a public API, protocol,\nconfig, migration, storage, security, platform, default, prompt-byte, generated-output\n(SKILL.md golden), package, release or architecture contract; call order when order is\nobservable; source inspection when it is the cheapest independent guard; anything\nreachable from the package entrypoint (`package.json` exports/main, index re-exports).\nStatic or slow is not a reason to delete.\n\nEvidence for each finding goes in `evidence` with the fields `detects` (what failure the\ntest can actually detect), `non_test_callers`, `search_command` and `stronger_proof`.\nFor the last two patterns, run the caller check for each symbol the diff adds or exports,\nonly when the symbol matches `^[A-Za-z_][A-Za-z0-9_]*$` (otherwise record \"caller check\nunavailable: unsupported symbol\"), with the symbol quoted, never interpolated unquoted:\n\n```bash\ngit grep -n -F -w -e '<symbol>' -- . ':!test/' ':!tests/' ':!spec/' ':!**/__tests__/**' ':!**/*.test.*' ':!**/*.spec.*' ':!**/*_test.*' ':!**/test_*.py'\n```\n\nRecord the command, exclusions and hit count, and mark the evidence \"grep-only\" (it cannot\nsee re-exports, dynamic dispatch or generated code). If the search fails, record: caller\ncheck unavailable: <command>. The finding stays INFORMATIONAL and nothing is proposed for\ndeletion; run the search by hand to complete the evidence. (see\n~/.claude/skills/gstack/docs/test-value-bar.md#caller-check-unavailable)\n\nSkip a test carrying `gstack:test-value keep reason=\"<why>\"` (any comment syntax). If any\nwere skipped, report their count and reasons as one INFORMATIONAL line. Emit no finding\nfor a test you reviewed and kept.\n\nRejection vocabulary, when a finding names why a new test fails the gate:\n`duplicate_protects`, `needs_seam`, `incomplete_card`, `no_credible_regression`,\n`covered_elsewhere`, `implementation_coupled`.\n\n### Regression test without red proof\n- The diff adds a test named or commented as a regression, and neither the commit history\n  nor the PR body shows it failing before the fix. A regression test that never\n  demonstrably failed proves the mock, not the fix. INFORMATIONAL; ask for the\n  fails-at-HEAD / passes-after-fix record.\n----\n     1\t\n     2\texport function processPayment(amount: number, currency: string) {\n     3\t  if (amount <= 0) throw new Error('Invalid amount');\n     4\t  if (currency !== 'USD' && currency !== 'EUR') throw new Error('Unsupported currency');\n     5\t  return { status: 'success', amount, currency };\n     6\t}\n     7\t\n     8\texport function refundPayment(paymentId: string, reason: string) {\n     9\t  if (!paymentId) throw new Error('Payment ID required');\n    10\t  if (!reason) throw new Error('Reason required');\n    11\t  return { status: 'refunded', paymentId, reason };\n    12\t}\n    13\t\n    14\t// coverage-read-evidence: 70e4e799-f2a2-455e-bd90-5104f81dd28f\n----\n     1\t\n     2\timport { describe, test, expect } from 'vitest';\n     3\timport { processPayment } from '../src/billing';\n     4\t\n     5\tdescribe('processPayment', () => {\n     6\t  test('processes valid payment', () => {\n     7\t    const result = processPayment(100, 'USD');\n     8\t    expect(result.status).toBe('success');\n     9\t  });\n    10\t  // GAP: no test for invalid amount\n    11\t  // GAP: no test for unsupported currency\n    12\t  // GAP: refundPayment not tested at all\n    13\t});\n    14\t\n    15\t// coverage-read-evidence: 5751c192-a564-47ec-8ef8-86757f97a26a\n----"
  },
  "files": {
    "source": "\nexport function processPayment(amount: number, currency: string) {\n  if (amount <= 0) throw new Error('Invalid amount');\n  if (currency !== 'USD' && currency !== 'EUR') throw new Error('Unsupported currency');\n  return { status: 'success', amount, currency };\n}\n\nexport function refundPayment(paymentId: string, reason: string) {\n  if (!paymentId) throw new Error('Payment ID required');\n  if (!reason) throw new Error('Reason required');\n  return { status: 'refunded', paymentId, reason };\n}\n\n// coverage-read-evidence: 70e4e799-f2a2-455e-bd90-5104f81dd28f\n",
    "tests": "\nimport { describe, test, expect } from 'vitest';\nimport { processPayment } from '../src/billing';\n\ndescribe('processPayment', () => {\n  test('processes valid payment', () => {\n    const result = processPayment(100, 'USD');\n    expect(result.status).toBe('success');\n  });\n  // GAP: no test for invalid amount\n  // GAP: no test for unsupported currency\n  // GAP: refundPayment not tested at all\n});\n\n// coverage-read-evidence: 5751c192-a564-47ec-8ef8-86757f97a26a\n"
  },
  "finalOutput": "I applied the testing specialist checklist on its own, with no other specialists and no fixes. I read the two files directly: `src/billing.ts` and `test/billing.test.ts`.\n\n```text\nsrc/billing.ts\nprocessPayment(amount, currency)\n├── valid USD happy path, processPayment(100, 'USD') returns status 'success' [OK]\n├── valid EUR happy path (second allowed currency, line 4) [GAP]\n├── returned amount/currency echo values (line 5, only status asserted) [GAP]\n├── amount <= 0 guard throws 'Invalid amount' (line 3, incl. 0 and negative boundary) [GAP]\n└── unsupported currency guard throws 'Unsupported currency' (line 4) [GAP]\nrefundPayment(paymentId, reason)\n├── refund success path returns status 'refunded' (line 11) [GAP]\n├── empty paymentId guard throws 'Payment ID required' (line 9) [GAP]\n└── empty reason guard throws 'Reason required' (line 10) [GAP]\nLegend: [OK] tested [GAP] no test\n```\n\n**Coverage summary**\n- `processPayment`: 1 of 5 paths covered. Only the valid USD call is tested, and it checks only `status`.\n- `refundPayment`: 0 of 3 paths covered. The test file doesn't import it.\n- Overall: 1 of 8 paths covered (about 12%). One function is partly tested and one has no tests. None of the three guard clauses that throw errors has a test.\n\n**Findings**\n{\"severity\":\"CRITICAL\",\"confidence\":10,\"path\":\"src/billing.ts\",\"line\":8,\"category\":\"testing\",\"summary\":\"refundPayment has zero test coverage: it is not imported in test/billing.test.ts, so its success path and both guard clauses (empty paymentId, empty reason) are untested\",\"fix\":\"Import refundPayment and add tests: refundPayment('pay_1','dup') returns {status:'refunded',paymentId:'pay_1',reason:'dup'}; refundPayment('', 'x') throws 'Payment ID required'; refundPayment('pay_1','') throws 'Reason required'\",\"fingerprint\":\"src/billing.ts:8:testing\",\"specialist\":\"testing\"}\n{\"severity\":\"CRITICAL\",\"confidence\":10,\"path\":\"src/billing.ts\",\"line\":3,\"category\":\"testing\",\"summary\":\"Invalid-amount guard (amount <= 0) has no negative-"
}