Commit Graph

59 Commits

Author SHA1 Message Date
copilot-swe-agent[bot] 2b720f3d88 Remove step-level timeouts to allow artifact downloads to complete
Co-authored-by: ggman12 <17393221+ggman12@users.noreply.github.com>
2026-02-16 19:38:27 +00:00
copilot-swe-agent[bot] e65da2cd95 Fix YAML property ordering for timeout-minutes
Co-authored-by: ggman12 <17393221+ggman12@users.noreply.github.com>
2026-02-16 17:50:18 +00:00
copilot-swe-agent[bot] 7ea815732c Add timeout and retry logic to adsb-reduce artifact download step
Co-authored-by: ggman12 <17393221+ggman12@users.noreply.github.com>
2026-02-16 17:49:41 +00:00
copilot-swe-agent[bot] 7fd5df8777 Initial plan 2026-02-16 17:44:39 +00:00
ggman12 a8b2b66952 fix .csv to .csv.gz transition openairframes-2026-02-17-develop openairframes-2026-02-17-main openairframes-2026-02-16-main openairframes-2026-02-16-develop 2026-02-15 19:08:51 -05:00
ggman12 3f38263a0c stop depue that destroys previous days 2026-02-15 17:55:16 -05:00
ggman12 1a553d5f44 use date of file instead of min timestamp 2026-02-15 16:44:09 -05:00
ggman12 405855c566 deal with whole schema 2026-02-15 16:43:00 -05:00
ggman12 4e81dde201 fix date parsing 2026-02-15 14:55:32 -05:00
ggman12 fde8ef029c update csv writing to handle empty data. Save space with higher gzip compression 2026-02-15 14:14:54 -05:00
ggman12 18ab51bd60 update naming 2026-02-15 13:45:03 -05:00
ggman12 83b9d2a76d write gzip 2026-02-15 13:41:09 -05:00
ggman12 8874619ab0 write gzip 2026-02-15 13:41:02 -05:00
ggman12 823f291728 fix errors in daily release due to new .gz file 2026-02-15 13:21:51 -05:00
ggman12 982011b36f end of year check 2026-02-14 22:42:32 -05:00
ggman12 1b15e43669 use .csv.gz 2026-02-14 22:22:14 -05:00
ggman12 f17adc4574 remvoe aws worker, reducer 2026-02-14 22:21:14 -05:00
ggman12 6a250a63fb fix None value comparision openairframes-2026-02-15-main openairframes-2026-02-15-develop 2026-02-14 20:21:32 -05:00
ggman12 9e24fcbc63 update integrity checker. Hopefully solve issue. 2026-02-14 19:56:25 -05:00
ggman12 8ce04f1f83 Revert "update for historical run"
This reverts commit ccf55b2308.
2026-02-14 18:44:21 -05:00
ggman12 9441761ac9 use temp release too. 2026-02-14 18:43:25 -05:00
ggman12 ccf55b2308 update for historical run 2026-02-14 15:57:16 -05:00
ggman12 76eaf118ef add run_local.py 2026-02-14 15:54:36 -05:00
ggman12 0fcbad0fbc let mictronics retry 2026-02-14 15:07:08 -05:00
ggman12 0c7484e7bf create_daily_microtonics release openairframes-2026-02-14-develop 2026-02-13 22:19:02 -05:00
ggman12 8c60ac611d create daily adsbexchange database snapshot release 2026-02-13 22:19:02 -05:00
ggman12 145f1006be update template openairframes-2026-02-13-develop 2026-02-13 12:12:24 -05:00
ggman12 f5465f0552 update .github/workflows/update-community-prs.yaml 2026-02-13 12:00:10 -05:00
ggman12 17098ae39a fix update-community-prs.yaml 2026-02-13 11:52:53 -05:00
ggman12 6f6b65780a update community_submission.yaml. update Readme.md 2026-02-13 11:49:18 -05:00
ggman12 4015a5fcf1 OpenAirframes 1.0 2026-02-13 11:37:31 -05:00
JG f9e04337ae Merge pull request #5 from PlaneQuery/develop
FIX: trigger for planequery-aircraft daily release workflow. Update contributions issue template.
planequery-aircraft-2026-02-12-main
2026-02-12 10:42:47 -05:00
ggman12 1348e1f3a0 Merge branch 'main' into develop 2026-02-12 10:41:26 -05:00
ggman12 b349c01d31 FIX: trigger for planequery-aircraft daily release workflow. Update contributions issue template. 2026-02-12 10:26:05 -05:00
JG a98175bc6c Merge pull request #3 from PlaneQuery/develop
Develop to main new historical adsb workflow. Community Submission updates.
2026-02-11 23:42:40 -05:00
ggman12 953a3647df remove process historical-faa github workflow 2026-02-11 23:41:42 -05:00
ggman12 e5c99b611c make a histoircla runner for adsb 2026-02-11 23:41:42 -05:00
ggman12 4e803dbb45 remove confirmations 2026-02-11 23:41:42 -05:00
JG 59c2aab5c7 Merge pull request #2 from PlaneQuery/develop
develop to main FEATURE: Add contributions framework. Fix and improve daily adsb release
2026-02-11 23:24:01 -05:00
ggman12 722bcdf791 FEATURE: Add contributions framework. Fix and improve daily adsb release using Github actions for map reduce. 2026-02-11 23:22:46 -05:00
ggman12 27da93801e FEATURE: add historical adsb aircraft data and update daily adsb aircraft data derivation.
add clickhouse_connect

use 32GB

update to no longer do df.copy()

Add planequery_adsb_read.ipynb

INCREASE: update Fargate task definition to 16 vCPU and 64 GB memory for improved performance on large datasets

update notebook

remove print(df)

Ensure empty strings are preserved in DataFrame columns

check if day has data for adsb

update notebook
2026-02-11 13:58:56 -05:00
JG b94bfdc575 Merge pull request #1 from PlaneQuery/import/af-klm-fleet
af-klm-fleet from iclems
planequery-aircraft-2026-02-09 planequery-aircraft-2026-02-05 planequery-aircraft-2026-02-06 planequery-aircraft-2026-02-07 planequery-aircraft-2026-02-08 planequery-aircraft-2026-02-10 planequery-aircraft-2026-02-11 planequery-aircraft-2026-02-12-develop
2026-02-04 17:51:46 -05:00
ggman12 c90bdada76 delete air-france folder 2026-02-04 17:49:25 -05:00
ggman12 921cbefb6e Add 'af-klm-fleet/' from commit 'b1dd01c27eccc8ba620994b6ae0df78a37075f3a'
git-subtree-dir: af-klm-fleet
git-subtree-mainline: 85a3db4dd0
git-subtree-split: b1dd01c27e
2026-02-04 17:47:47 -05:00
Clément Wehrung b1dd01c27e Fix API rate limit risk 2026-02-04 23:27:15 +01:00
Clément Wehrung 2282e1197f Auto-update fleet data - 2026-02-04 2026-02-04 23:25:27 +01:00
Clément Wehrung ea9c095f91 Fix generate README / License 2026-02-04 23:25:18 +01:00
Clément Wehrung 4eb2b9ce0b Auto-update fleet data - 2026-02-04 2026-02-04 23:21:27 +01:00
Clément Wehrung 23ef72100f Merge branch 'main' of github.com:iclems/af-klm-fleet 2026-02-04 23:20:04 +01:00
Clément Wehrung bfb22670ba Added cron update 2026-02-04 23:19:02 +01:00