CalvinBackup/OpenAirframes
1
0
Fork 0
mirror of https://github.com/PlaneQuery/OpenAirframes.git synced 2026-04-23 19:46:09 +02:00
Code Issues Packages Projects Releases Wiki Activity
96 Commits 13 Branches 147 Tags
jonah/fix-historical-proper
Commit Graph

96 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
ggman12 4703383648 add hisotircal-adsb-run.yaml 2026-02-17 17:55:49 -05:00
ggman12 53633bc09f use v6 for python 2026-02-17 17:39:54 -05:00
ggman12 f6897c1b72 updates to .github/workflows/historical-adsb.yaml 2026-02-17 17:36:47 -05:00
ggman12 19f3c5b63c Add main pipeline for processing ADS-B data with date argument 2026-02-17 17:27:44 -05:00
ggman12 b2f6a751fa Refactor concat_parquet_to_final.py to accept date as an argument and streamline file handling 2026-02-17 17:27:41 -05:00
ggman12 88b00c1cf6 add .snapshots to .gitignore 2026-02-17 17:27:28 -05:00
ggman12 b6bf915cec Filter rows by date in compress_parquet_part function 2026-02-17 16:59:09 -05:00
ggman12 6306aade16 Make global NUMBER_PARTS. remove print. 2026-02-17 16:42:59 -05:00
ggman12 9c54d9f1e4 remove print 2026-02-17 16:40:25 -05:00
ggman12 e8707ab853 get rid of unused code 2026-02-17 16:18:26 -05:00
ggman12 ca5cb23a4d use date folder 2026-02-17 16:16:46 -05:00
ggman12 121dccf26c concat file 2026-02-17 16:16:41 -05:00
ggman12 db98b3021a change to use path 2026-02-17 16:16:35 -05:00
ggman12 94cf50ac3a make return consistant 2026-02-17 16:09:09 -05:00
ggman12 ac177e8025 output .csv too 2026-02-17 16:09:01 -05:00
ggman12 6b7068bc84 works 2026-02-17 15:51:20 -05:00
ggman12 70ec797535 works 2026-02-17 15:46:07 -05:00
ggman12 1afe2bed4e remvoe code from src/adsb/process_icao_chunk.py 2026-02-17 15:42:45 -05:00
ggman12 d3c52266e5 fix tar corrruption 2026-02-17 15:42:35 -05:00
ggman12 c0dca14b83 remove unused code 2026-02-17 14:57:29 -05:00
ggman12 1fc4a94743 do only a single day instead of multiple 2026-02-17 14:23:30 -05:00
ggman12 f29abad52a output to parted tar.gz 2026-02-17 14:10:01 -05:00
ggman12 6eb84a894b add notebooks/whatever.ipynb to .gitignore 2026-02-17 12:48:59 -05:00
ggman12 0c81490513 make it single day 2026-02-17 12:48:28 -05:00
ggman12 11ed7e597d delete unused code 2026-02-17 12:48:01 -05:00
ggman12 24c0fc970c use exclusive end_date 2026-02-17 12:47:44 -05:00
ggman12 c12e855b5a change from 7 days to 1 2026-02-16 20:36:20 -05:00
ggman12 b55690638c feat: implement download and concatenate script for workflow artifacts 2026-02-16 20:34:22 -05:00
ggman12 dcee136f09 refactor: update historical-adsb script to use 15-day chunks and improve argument handling 2026-02-16 20:14:04 -05:00
ggman12 035748fc61 skip using base release in run_local.py 2026-02-16 18:26:53 -05:00
ggman12 13432068e6 src/adsb/run_local.py works 2026-02-16 17:45:31 -05:00
ggman12 9cb4c5045b remove compression from github action 2026-02-16 17:41:15 -05:00
ggman12 343a391a3f change default chunk_days from 7 to 3 2026-02-16 16:35:05 -05:00
ggman12 2bc45ff6a4 increase retry_delay to 5 minutes. 2026-02-16 15:35:02 -05:00
ggman12 03291d93a8 add scripts/run_historical_adsb_action.py 2026-02-16 14:54:25 -05:00
ggman12 5883b459ac fix bug with no dupliacte icaos across days 2026-02-15 21:08:17 -05:00
ggman12 f8ba66375b preserve time 2026-02-15 21:08:03 -05:00
ggman12 7a62faecef sort by time in end 2026-02-15 20:33:06 -05:00
ggman12 9964ce576b slight update for compress by day 2026-02-15 20:32:33 -05:00
ggman12 be33fd2eaf compress by day 2026-02-15 19:59:50 -05:00
ggman12 2b2095700f use chunks in run_local 2026-02-15 19:53:09 -05:00
ggman12 a8b2b66952 fix .csv to .csv.gz transition openairframes-2026-02-17-develop openairframes-2026-02-17-main openairframes-2026-02-16-main openairframes-2026-02-16-develop 2026-02-15 19:08:51 -05:00
ggman12 3f38263a0c stop depue that destroys previous days 2026-02-15 17:55:16 -05:00
ggman12 1a553d5f44 use date of file instead of min timestamp 2026-02-15 16:44:09 -05:00
ggman12 405855c566 deal with whole schema 2026-02-15 16:43:00 -05:00
ggman12 4e81dde201 fix date parsing 2026-02-15 14:55:32 -05:00
ggman12 fde8ef029c update csv writing to handle empty data. Save space with higher gzip compression 2026-02-15 14:14:54 -05:00
ggman12 18ab51bd60 update naming 2026-02-15 13:45:03 -05:00
ggman12 83b9d2a76d write gzip 2026-02-15 13:41:09 -05:00
ggman12 8874619ab0 write gzip 2026-02-15 13:41:02 -05:00