From 5a2be1a795b0d11d128a35c7ee134ea49a834ada Mon Sep 17 00:00:00 2001 From: garrytan Date: Fri, 2 Oct 2026 23:50:32 +0000 Subject: [PATCH] testing: coverage trace and test generation follow the value bar (s03 M2, s03 M3) - M2: 'Every branch in this diagram needs a test' / 'Each step in the journey needs a test' contradicted COVERAGE_GOAL ('Test count is not a goal') in the same rendered audit. Branches and journey steps now need coverage that would catch a regression; the value bar decides new test vs extension vs covered. - M3: 'Mock all external dependencies' -> the smallest native test, mocking only services unrelated to the behavior (matches the QA handoff in qa.ts and the audit's own E2E routing). No test pins the removed text. expected red until the regeneration commit --- scripts/resolvers/testing.ts | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/resolvers/testing.ts b/scripts/resolvers/testing.ts index 0fda3fb17..36dd64554 100644 --- a/scripts/resolvers/testing.ts +++ b/scripts/resolvers/testing.ts @@ -341,7 +341,7 @@ context. Base the diagram on that read. - Every call to another function (trace into it — does IT have untested branches?) - Every edge: what happens with null input? Empty array? Invalid type? -This is the critical step — you're building a map of every line of code that can execute differently based on input. Every branch in this diagram needs a test.`); +This is the critical step — you're building a map of every line of code that can execute differently based on input. Every branch in this diagram needs coverage that would catch a real regression; the test value bar below decides whether that is a new test, an extension of an existing one, or already covered.`); // ── User flow coverage (shared) ── sections.push(` @@ -349,7 +349,7 @@ This is the critical step — you're building a map of every line of code that c Code coverage isn't enough — you need to cover how real users interact with ${mode === 'plan' ? 'the selected target. For each existing or proposed feature' : 'the changed code. For each changed feature'}, think through: -- **User flows:** What sequence of actions does a user take that touches this code? Map the full journey (e.g., "user clicks 'Pay' → form validates → API call → success/failure screen"). Each step in the journey needs a test. +- **User flows:** What sequence of actions does a user take that touches this code? Map the full journey (e.g., "user clicks 'Pay' → form validates → API call → success/failure screen"). Each step in the journey needs coverage. - **Interaction edge cases:** What happens when the user does something unexpected? - Double-click/rapid resubmit - Navigate away mid-operation (back button, close tab, click another link) @@ -538,7 +538,7 @@ If test framework detected (or bootstrapped in Step 4): - Write the value card as a header comment in each generated or extended test. - Prioritize error handlers and edge cases first (happy paths are more likely already tested) - Read 2-3 existing test files to match conventions exactly -- Generate unit tests. Mock all external dependencies (DB, API, Redis). +- Generate the smallest native test for each path: unit tests for logic, mocking only services unrelated to the behavior under test (integration points where mocking hides real failures go to [→E2E]). - For paths marked [→E2E]: generate integration/E2E tests using the project's E2E framework (Playwright, Cypress, Capybara, etc.) - For paths marked [→EVAL]: generate eval tests using the project's eval framework, or flag for manual eval if none exists - Write tests that exercise the specific uncovered path with real assertions