fix(diff-scope): count Python test_*.py and nested test dirs as tests (#3335)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
garrytanandcapy-ai[bot] committed 2026-10-24 20:09:22 +00:00
1 parent 00100f1598
commit bd23d0327a
3 files changed
+45 -1

No files matched your search

+10
View File
@@ -1,5 +1,15 @@
# Changelog
## [Unreleased]
**`gstack-diff-scope` sees Python tests and nested test directories (#3335).** A branch that changed `components/orch/services/shared/tests/test_native_store_custody.py` reported `SCOPE_TESTS=false`, because the Tests rule only knew `*.test.*`-style suffixes and `test/`, `tests/`, `spec/` and `__tests__/` at the repository root. /ship and /review use that flag to route the testing specialist, so a backend change whose tests follow pytest's `test_*.py` convention skipped it. The test-bootstrap resolver already counted these files, so the two detectors disagreed.
### Fixed
- **`bin/gstack-diff-scope`**: a changed path is a test when it matches `TEST_PATH_RE`, the same regex the test-bootstrap resolver uses for its TESTFILES count (`scripts/resolvers/testing.ts`): test dirs at any depth, `test_*.py`, `tests.py`, `_test.<ext>`, `.test.`/`.spec.` JS and TS, `_spec.rb` and `*Test.java`/`.kt`. `conftest.py` counts too. The existing globs stay.
### Tests
- `test/diff-scope.test.ts` adds the issue's three paths, `pkg/test_store.py`, Django's `tests.py`, `conftest.py`, and nested `__tests__/` and `spec/` dirs; all eight fail on main. Controls that stay `SCOPE_TESTS=false`: `latest.py`, `contest_rules.py`, `attest_signer.py` and a `testing/` dir. A new test checks that `TEST_PATH_RE` is byte-identical to the resolver's regex, so the two can't drift again.
## [1.91.186.0] - 2026-10-24
**Approving an autoplan run no longer fails one time in ten because its run id looks like a card number.** Today's daily flaky-fix pass found `test/capy-captain-flow.test.ts` failing once and passing on the serial retry in 7 of the 57 Free Tests runs from 2026-10-21 to 2026-10-23, one of them a push to main (39878113535). Every failure had the same assertion: `gstack-approvals import` refused the approval row with "field ask: pii.cc". The default autoplan run id is `run-YYYYMMDDHHMMSS`, the import writes it into the row's ask and source, and the credit-card rule saw those 14 digits as a card number whenever they passed the Luhn check, which a timestamp does about one time in ten. A real user's approval import failed the same way.
+8
View File
@@ -42,6 +42,12 @@ MIGRATIONS=false
API=false
AUTH=false
# Test-file paths, the same rule the test-bootstrap resolver counts as
# TESTFILES (scripts/resolvers/testing.ts; test/diff-scope.test.ts keeps the two
# identical). Nested test dirs and Python's test_*.py / tests.py were missed by
# the root-anchored globs alone (#3335).
TEST_PATH_RE='(^|/)(tests?|spec|__tests__)/|(^|/)tests?\.py$|(^|/)test_[^/]+\.py$|_test\.(go|py|rb|ts|js|exs|dart)$|\.(test|spec)\.[jt]sx?$|_spec\.rb$|Test\.(java|kt)$'
_print_flags() {
echo "SCOPE_FRONTEND=$FRONTEND"
echo "SCOPE_BACKEND=$BACKEND"
@@ -126,7 +132,9 @@ for f in ${FILES_LIST[@]+"${FILES_LIST[@]}"}; do
case "$f" in
*.test.*|*.spec.*|*_test.*|*_spec.*) m_tests=true ;;
test/*|tests/*|spec/*|__tests__/*|cypress/*|e2e/*) m_tests=true ;;
conftest.py|*/conftest.py) m_tests=true ;;
esac
if [[ "$f" =~ $TEST_PATH_RE ]]; then m_tests=true; fi
# Docs
case "$f" in
+27 -1
View File
@@ -5,7 +5,7 @@
* the correct SCOPE_* variables are output.
*/
import { describe, test, expect, afterAll } from 'bun:test';
import { mkdtempSync, writeFileSync, mkdirSync, rmSync } from 'fs';
import { mkdtempSync, writeFileSync, mkdirSync, rmSync, readFileSync } from 'fs';
import { join } from 'path';
import { tmpdir } from 'os';
import { spawnSync } from 'child_process';
@@ -245,6 +245,20 @@ describe('glob classes (table-driven, #2526 + #2455)', () => {
{ name: 'plain backend', file: 'server.go', expects: ['SCOPE_BACKEND'], alsoFalse: ['SCOPE_FRONTEND'] },
{ name: 'docs', file: 'docs/guide.md', expects: ['SCOPE_DOCS'] },
{ name: 'prompts', file: 'app/services/prompt_builder.rb', expects: ['SCOPE_PROMPTS', 'SCOPE_BACKEND'] },
// Python tests and nested test dirs (#3335): the issue's three paths.
{ name: 'nested tests/test_*.py (#3335)', file: 'components/orch/services/shared/tests/test_native_store_custody.py', expects: ['SCOPE_TESTS', 'SCOPE_BACKEND'] },
{ name: 'second nested tests/test_*.py (#3335)', file: 'components/orch/services/shared/tests/test_infra_promoted_first_apply.py', expects: ['SCOPE_TESTS', 'SCOPE_BACKEND'] },
{ name: 'hyphenated service tests/test_*.py (#3335)', file: 'components/orch/services/api-gateway/tests/test_native_chat_checkpoint_store.py', expects: ['SCOPE_TESTS', 'SCOPE_BACKEND'] },
{ name: 'test_*.py outside a tests dir', file: 'pkg/test_store.py', expects: ['SCOPE_TESTS', 'SCOPE_BACKEND'] },
{ name: 'Django tests.py', file: 'shop/tests.py', expects: ['SCOPE_TESTS', 'SCOPE_BACKEND'] },
{ name: 'pytest conftest.py', file: 'services/api/conftest.py', expects: ['SCOPE_TESTS', 'SCOPE_BACKEND'] },
{ name: 'nested __tests__ dir', file: 'packages/ui/__tests__/render.ts', expects: ['SCOPE_TESTS', 'SCOPE_BACKEND'] },
{ name: 'nested spec dir', file: 'engines/billing/spec/invoice_checker.rb', expects: ['SCOPE_TESTS', 'SCOPE_BACKEND'] },
// Controls: a name that only contains "test" is not a test file.
{ name: 'latest.py is not a test', file: 'pkg/latest.py', expects: ['SCOPE_BACKEND'], alsoFalse: ['SCOPE_TESTS'] },
{ name: 'contest_rules.py is not a test', file: 'pkg/contest_rules.py', expects: ['SCOPE_BACKEND'], alsoFalse: ['SCOPE_TESTS'] },
{ name: 'testing/ dir is not a test dir', file: 'pkg/testing/fixtures_loader.py', expects: ['SCOPE_BACKEND'], alsoFalse: ['SCOPE_TESTS'] },
{ name: 'attest_*.py is not a test', file: 'pkg/attest_signer.py', expects: ['SCOPE_BACKEND'], alsoFalse: ['SCOPE_TESTS'] },
];
for (const row of TABLE) {
@@ -262,6 +276,18 @@ describe('glob classes (table-driven, #2526 + #2455)', () => {
}
});
describe('one test-path rule (#3335)', () => {
test('TEST_PATH_RE is byte-identical to the test-bootstrap TESTFILES regex', () => {
const script = readFileSync(SCRIPT, 'utf8');
const scriptRe = script.match(/^TEST_PATH_RE='([^']+)'$/m)?.[1];
const resolver = readFileSync(join(import.meta.dir, '..', 'scripts', 'resolvers', 'testing.ts'), 'utf8');
const resolverRe = resolver.match(/git ls-files \| grep -cE '([^']+)'/)?.[1]?.replace(/\\\\/g, '\\');
expect(scriptRe).toBeDefined();
expect(resolverRe).toBeDefined();
expect(scriptRe).toBe(resolverRe);
});
});
describe('exit-code contract (#2526)', () => {
test('clean tree, no changes → all false, exit 0, no SCOPE_ERROR', () => {
const dir = createRepo([]);