fix: make registry join failures visible instead of silent

- accept source ids containing digits and underscores; the previous pattern dropped
  openairframes_uk_caa_*.csv with no output, contradicting the documented promise that
  adding a source needs no change here
- reject a file whose source column disagrees with its filename, which would otherwise
  merge one registry into the union under another's label
- refuse duplicate source files, skip an empty optional source rather than aborting, and
  log every skipped file and the column names shared across sources
- correct the MIN_EXPECTED_ROWS comment, which described a 35k floor for a 1000 value

Generated-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Ashley Childress
2026-09-10 20:53:54 -04:00
parent 10062956a4
commit f78fa1c3ee
2 changed files with 35 additions and 7 deletions
+4 -2
View File
@@ -50,8 +50,10 @@ OWNER_ADDRESS_COLUMNS = {
FOOTER_RE = re.compile(r"\s*(\d+) rows selected\.\s*")
# Canada's register is ~35k aircraft. Any parse yielding less than this means the
# export was truncated upstream, which must not be published as a real snapshot.
# Floor, not an expectation: Canada's register is ~35k aircraft and carsownr is larger
# still, so 1000 only catches a grossly truncated export. The footer row-count check above
# is what actually validates the parse; this guards the case where the footer agrees with a
# near-empty body.
MIN_EXPECTED_ROWS = 1000