review: testing specialist treats negative assertions as decisions

A new assertion that something is absent records a behavior removal and
needs cited intent. A rename or removal in the diff can empty a standing
negative assertion so it passes forever; the specialist now greps the suite
for negative assertions naming the old value.

Adopted from the testing-specialist part of a community PR; the scope-check
hunk targeted text that no longer exists.

Co-authored-by: EmilianoU26 <314541067+EmilianoU26@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
committed 2026-10-08 14:53:02 +00:00
1 parent 0668443f36
commit 7d88f13f58
2 files changed
+23 -1

No files matched your search

+11
View File
@@ -58,6 +58,17 @@ red tests. Unit tests suit logic; real integration/E2E tests protect boundaries
- Unicode and special characters in user-facing inputs
- Concurrent access patterns with no race-condition test
### Negative Assertions
- A new assertion that something is absent (`not.toContain`, `not in`, `assertNotIn`,
`pytest.raises` on a call that used to succeed) records a behavior removal, not coverage.
Cite the spec line, plan item or PR text that authorizes the removal; with none, report it
as a product decision shipped as a test.
- A rename or removal in this diff can empty a standing negative assertion: a test asserting
the old token is absent now passes forever and guards nothing. For each identifier, flag,
heading or string the diff renames or deletes, grep the test suite for negative assertions
naming the old value, including inside tests that assert something else, and report each
as a coverage gap whose fix names the new value.
### Test Isolation Violations
- Tests sharing mutable state (class variables, global singletons, DB records not cleaned up)
- Order-dependent tests (pass in sequence, fail when randomized)
+12 -1
View File
@@ -7,7 +7,7 @@ import { generatePlanCompletionAuditReview, generatePlanCompletionAuditShip, gen
import { generateQAReview } from '../scripts/resolvers/qa';
import { generateConfidenceCalibration } from '../scripts/resolvers/confidence';
import { HOST_PATHS, type TemplateContext } from '../scripts/resolvers/types';
import { expectMentions } from './helpers/prompt-structure';
import { between, compact, expectMentions, expectTokens } from './helpers/prompt-structure';
const root = join(import.meta.dir, '..');
const skill = readFileSync(join(root, 'review/SKILL.md.tmpl'), 'utf8');
@@ -466,3 +466,14 @@ test('ship and review count unverified or unavailable outside reviews as missing
expect(read(rel)).toMatch(/`unverified` or `unavailable` is (listed as )?missing coverage[^.]*never (as )?(a )?pass/i);
}
});
test('testing specialist treats negative assertions as decisions and catches ones a rename emptied', () => {
const specialist = between(readFileSync(join(root, 'review/specialists/testing.md'), 'utf8'), '### Negative Assertions', '\n### ');
expectTokens(specialist, ['not.toContain', 'assertNotIn'], 'testing specialist');
expectMentions(compact(specialist), [
['new', 'absent', 'removal', 'not coverage'],
['authorizes', 'removal'],
['rename', 'negative assertion', 'passes forever'],
['grep', 'negative assertions', 'old value'],
], 'testing specialist');
});