Commit Graph
19 Commits
Author SHA1 Message Date
Ashley Childress 1bbaba909e feat: build each registry source on its own thread and join them
- add a reusable registry-source workflow so every source runs in parallel rather than as
  a hand-written job; adding a registry becomes one matrix entry plus one script
- add build_registry.py to align the per-source CSVs on the union of columns and emit a
  single table discriminated by the source column
- reindex each frame to the union before concatenating, so a source missing a column
  yields an empty cell rather than a shifted row

Generated-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 21:15:11 -04:00
Ashley Childress 2c5e8b9621 docs: cut AGENTS.md back to operative constraints
- drop narrative rationale, restated rules and background a reader can grep for
- one rule per line; the file is read by a model about to act, not by a person catching up
- 151 lines to 85

Generated-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 21:12:22 -04:00
Ashley Childress 1b6de19afb refactor: name Transport Canada columns in the FAA vocabulary
- registrant_* rather than owner_*, status rather than registration_status, so both
  registries describe the same concept with the same column name
- registrant_zip_code carries the Canadian postal code: a union table needs one column per
  concept, not one per country's vocabulary
- set source="TC", matching the discriminator the FAA frame already carries
- raises the column names shared with the FAA frame from 9 to 21

Generated-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 21:12:22 -04:00
Ashley Childress fce0b8d18c feat: redistribute Canadian owner addresses to match the FAA asset
- publish street, city, postal code and care-of, which the FAA asset already carries as
  registrant_* for 99.7% of US registrants; dropping them here left one repository with
  two different postures on the same class of data
- take the address from the single MAIL_RECIPIENT row rather than merging across parties,
  since a co-owned mark lists several people in different cities

Generated-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 21:12:00 -04:00
Ashley Childress bd78cc5b2c docs: correct licence and ownership claims in NOTICE and AGENTS.md
- stop asserting US public-domain status for the derived FAA CSV; section 105 covers the
  government's own work, not this repository's derivative
- name no licence instrument for Transport Canada, which does not publish one on the
  download page, rather than citing one that cannot be verified
- state that MIT covers code only, resolving a contradiction with AGENTS.md
- lead the source-eligibility rule with the bilateral-permission bar; Taiwan is OGDL
  licensed and excluded for that reason, not for commercial terms

Generated-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 21:12:00 -04:00
Ashley Childress 9a1b828d3c fix: stop a failed Transport Canada build from erasing release history
- fall back to a single-day rebuild only on FileNotFoundError; a rate limit, schema change
  or truncated download previously took the same path and republished one day as the whole
  dataset, which the next run then read back as its base
- move the monotonic download_date assert out of the try so corruption cannot select the
  destructive branch
- walk back through releases like the ADS-B reader, so one missing optional asset does not
  strand the accumulation
- authenticate release reads and verify downloaded asset size
- read the previous CSV with keep_default_na=False so literal NA values round-trip
- write the download atomically and reject non-zip responses

Generated-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 21:12:00 -04:00
Ashley Childress f5423724bc fix: validate the CCARCS parse and correct owner aggregation
- reject Mode S fields that are not 24 binary digits; a short field zero-padded into a
  plausible address belonging to a different aircraft
- check the "N rows selected." footer against the parsed row count and enforce a row floor,
  so an upstream short export fails instead of publishing as a smaller register
- prefer ACTIVE_FLAG "A" parties but fall back to all: 1,932 Registered marks carry only "I"
  rows, and those are the MAIL_RECIPIENT, so filtering on "A" alone drops real owners
- count distinct owner names rather than rows, fixing 154 marks labelled Co-owner in error
- match the spool footer by pattern instead of a brittle ragged-row count

Generated-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 21:12:00 -04:00
Ashley Childress 705a97690f docs: record the attribution and redistribution rules in AGENTS.md
- attribution is a licence condition and NOTICE must travel with release assets
- a public licence travels to this project; a bilateral permission does not
- non-commercial-only registries are incompatible with the MIT-licensed releases

Generated-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 21:12:00 -04:00
Ashley Childress 2711b2d0f8 feat: build a daily Transport Canada aircraft register release
- parse the headerless latin1 CCARCS export against its declared column layout
- derive transponder_code_hex from the 24-bit Mode S binary, populated for all 34,913 rows
- expand marks to C- and vintage CF- registrations and drop owner mailing addresses
- mirror the FAA build: same concat-with-latest-release dedup and output conventions

Generated-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 21:11:59 -04:00
Ashley Childress a735616201 docs: add NOTICE for redistributed data source terms
- carry the two Government of Canada notices the open licence requires verbatim
- record FAA public-domain provenance and unassessed third-party republished assets
- LICENSE covers code only; release data carries its own conditions

Generated-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 21:11:59 -04:00
Ashley Childress e0725bde52 docs: add AGENTS.md with CLAUDE.md pointing at it
- record only what the repo cannot reveal on inspection: intent, traps, prohibitions
- cover injection rules, invocation modes, silent-corruption couplings and dead paths
- flag warts that need an owner decision rather than a silent fix

Generated-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 17:46:42 -04:00
Ashley Childress df4a65e5c4 docs: correct comments that contradict the code
- part id is 0-indexed in both --help and the loader docstring, matching the matrix
- daily release cron comment said 6:00pm while the expression fires at 06:00 UTC

Generated-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 17:46:42 -04:00
Ashley Childress 6a6dc090bd chore: remove dead combine_historical_faa module
- module called an undefined master_txt_to_releasable_csv and could never run
- nothing in the repo imported or referenced it

Generated-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 17:46:42 -04:00
Ashley Childress f16ff3f4fd fix: open community PRs against the repository default branch
- query the repo for its default branch instead of assuming main
- use that branch as both the fork point and the pull request base

Generated-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 17:46:42 -04:00
Ashley Childress ba97448365 fix: resolve the community schema version at runtime
- write through get_schema_path() instead of a literal v1 filename
- drop the unreferenced SCHEMA_PATH backwards-compatibility shim
- widen the community PR trigger to schemas/** so a version bump still fires it

Generated-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 17:46:42 -04:00
Ashley Childress 3023c5015e fix: exit non-zero when the ADS-B concat step produces nothing
- fail when no parquet parts exist and --concat_with_latest_csv is not set
- keep the fallback path that re-releases the latest CSV when adsb.lol is late
- correct a log line that reported "no parquet files" on the populated branch

Generated-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 17:46:42 -04:00
Ashley Childress bcbd9f2985 refactor: define the released ADS-B column order in one place
- replace two independent copies with FINAL_COLUMN_ORDER
- polars concatenates by position, so a forked copy corrupted releases silently

Generated-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 17:46:42 -04:00
Ashley Childress d26535a2bc fix: treat HTTP 404 as terminal when listing adsb.lol releases
- stop retrying a nonexistent repo 10 times at 5-minute intervals
- cut the dead-end Dec-31 next-year probe from ~45 minutes to under a second
- leave 403, 429, 5xx and network errors on the existing retry path

Generated-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 17:46:42 -04:00
Ashley Childress f228d16c60 fix: pass issue body through env instead of a shell heredoc
- prevent command injection from a crafted issue body closing the heredoc early
- drop the debug step that echoed untrusted input for no operational value
- read body and issue number from env in the community validation workflow

Generated-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 17:46:41 -04:00