CalvinBackup/gstack
1
0
Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-05-02 03:35:09 +02:00
Code Issues Packages Projects Releases Wiki Activity

fix: strengthen E2E assertions for coverage audit tests

Browse Source 
The coverage audit E2E tests (ship + review) were only asserting
exitReason === 'success' and readCalls > 0 — they passed even
if the agent produced no coverage diagram. Add assertion that
the output contains either GAP or TESTED markers.

Found during /review.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-03-20 07:58:44 -07:00
parent c286a09793
commit 451d7c2fc1
1 changed files with 14 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test/skill-e2e.test.ts
+14 -6
View File
@@ -2770,14 +2770,18 @@ Output the diagram directly.`,
// Check output contains coverage diagram elements
const output = result.output || '';
const hasGap = output.includes('GAP') || output.includes('gap') || output.includes('NO TEST');
const hasTested = output.includes('TESTED') || output.includes('tested') || output.includes('✓');
const hasCoverage = output.includes('COVERAGE') || output.includes('coverage') || output.includes('paths tested');
const outputLower = output.toLowerCase();
const hasGap = outputLower.includes('gap') || outputLower.includes('no test');
const hasTested = outputLower.includes('tested') || output.includes('✓') || output.includes('★');
const hasCoverage = outputLower.includes('coverage') || outputLower.includes('paths tested');
console.log(`Output has GAP markers: ${hasGap}`);
console.log(`Output has TESTED markers: ${hasTested}`);
console.log(`Output has coverage summary: ${hasCoverage}`);
// The agent MUST produce a coverage diagram with gap and tested markers
expect(hasGap || hasTested).toBe(true);
// At minimum, the agent should have read the source and test files
const readCalls = result.toolCalls.filter(tc => tc.tool === 'Read');
expect(readCalls.length).toBeGreaterThan(0);
@@ -2836,14 +2840,18 @@ Output the diagram directly.`,
// Check output contains coverage diagram elements
const output = result.output || '';
const hasGap = output.includes('GAP') || output.includes('gap') || output.includes('NO TEST');
const hasTested = output.includes('TESTED') || output.includes('tested') || output.includes('✓');
const hasCoverage = output.includes('COVERAGE') || output.includes('coverage') || output.includes('paths tested');
const outputLower = output.toLowerCase();
const hasGap = outputLower.includes('gap') || outputLower.includes('no test');
const hasTested = outputLower.includes('tested') || output.includes('✓') || output.includes('★');
const hasCoverage = outputLower.includes('coverage') || outputLower.includes('paths tested');
console.log(`Output has GAP markers: ${hasGap}`);
console.log(`Output has TESTED markers: ${hasTested}`);
console.log(`Output has coverage summary: ${hasCoverage}`);
// The agent MUST produce a coverage diagram with gap and tested markers
expect(hasGap || hasTested).toBe(true);
// At minimum, the agent should have read the source and test files
const readCalls = result.toolCalls.filter(tc => tc.tool === 'Read');
expect(readCalls.length).toBeGreaterThan(0);