mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-20 15:13:47 +01:00
review: testing specialist treats negative assertions as decisions
A new assertion that something is absent records a behavior removal and needs cited intent. A rename or removal in the diff can empty a standing negative assertion so it passes forever; the specialist now greps the suite for negative assertions naming the old value. Adopted from the testing-specialist part of a community PR; the scope-check hunk targeted text that no longer exists. Co-authored-by: EmilianoU26 <314541067+EmilianoU26@users.noreply.github.com> Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
1 parent
0668443f36
commit
7d88f13f58
2 files changed
+23
-1
No files matched your search
@@ -58,6 +58,17 @@ red tests. Unit tests suit logic; real integration/E2E tests protect boundaries
|
||||
- Unicode and special characters in user-facing inputs
|
||||
- Concurrent access patterns with no race-condition test
|
||||
|
||||
### Negative Assertions
|
||||
- A new assertion that something is absent (`not.toContain`, `not in`, `assertNotIn`,
|
||||
`pytest.raises` on a call that used to succeed) records a behavior removal, not coverage.
|
||||
Cite the spec line, plan item or PR text that authorizes the removal; with none, report it
|
||||
as a product decision shipped as a test.
|
||||
- A rename or removal in this diff can empty a standing negative assertion: a test asserting
|
||||
the old token is absent now passes forever and guards nothing. For each identifier, flag,
|
||||
heading or string the diff renames or deletes, grep the test suite for negative assertions
|
||||
naming the old value, including inside tests that assert something else, and report each
|
||||
as a coverage gap whose fix names the new value.
|
||||
|
||||
### Test Isolation Violations
|
||||
- Tests sharing mutable state (class variables, global singletons, DB records not cleaned up)
|
||||
- Order-dependent tests (pass in sequence, fail when randomized)
|
||||
|
||||
@@ -7,7 +7,7 @@ import { generatePlanCompletionAuditReview, generatePlanCompletionAuditShip, gen
|
||||
import { generateQAReview } from '../scripts/resolvers/qa';
|
||||
import { generateConfidenceCalibration } from '../scripts/resolvers/confidence';
|
||||
import { HOST_PATHS, type TemplateContext } from '../scripts/resolvers/types';
|
||||
import { expectMentions } from './helpers/prompt-structure';
|
||||
import { between, compact, expectMentions, expectTokens } from './helpers/prompt-structure';
|
||||
|
||||
const root = join(import.meta.dir, '..');
|
||||
const skill = readFileSync(join(root, 'review/SKILL.md.tmpl'), 'utf8');
|
||||
@@ -466,3 +466,14 @@ test('ship and review count unverified or unavailable outside reviews as missing
|
||||
expect(read(rel)).toMatch(/`unverified` or `unavailable` is (listed as )?missing coverage[^.]*never (as )?(a )?pass/i);
|
||||
}
|
||||
});
|
||||
|
||||
test('testing specialist treats negative assertions as decisions and catches ones a rename emptied', () => {
|
||||
const specialist = between(readFileSync(join(root, 'review/specialists/testing.md'), 'utf8'), '### Negative Assertions', '\n### ');
|
||||
expectTokens(specialist, ['not.toContain', 'assertNotIn'], 'testing specialist');
|
||||
expectMentions(compact(specialist), [
|
||||
['new', 'absent', 'removal', 'not coverage'],
|
||||
['authorizes', 'removal'],
|
||||
['rename', 'negative assertion', 'passes forever'],
|
||||
['grep', 'negative assertions', 'old value'],
|
||||
], 'testing specialist');
|
||||
});
|
||||
Reference in new issue
Block a user