Compare commits

..

10 Commits

Author SHA1 Message Date
BigBodyCobain 03b8053617 feat(flights): cumulative fuel burned + CO2 emitted per flight
Pre-fix the emissions tooltip only showed the per-hour *rate* — what most
users actually want is the cumulative *amount* burned. This adds running
totals computed by multiplying the model-based rate by the elapsed
observation time since we first saw the airframe.

New module ``flight_observations.py``:
* Tracks first_seen_at + last_seen_at per icao24 hex.
* Re-opens a fresh session when an aircraft is unseen for > 15 min
  (treated as a new flight — landed and took off, or transited a dead
  zone). Prevents the cumulative counter from resetting mid-flight if
  the trail-rendering cache prunes the trail.
* Clamps elapsed time to 24h max so clock skew can't produce comically
  large numbers.
* Pruned every 5 min via a new scheduler job (mirrors ais_prune cadence).

flights.py + military.py emission enrichment now also attaches:
* observed_seconds — how long we've been tracking this airframe.
* fuel_gallons_burned — rate * elapsed_h.
* co2_kg_emitted — rate * elapsed_h.

The existing per-hour rate fields stay in the dict for backward compat
and are shown as small secondary context in the tooltip.

Frontend EmissionsEstimateBlock (NewsFeed.tsx) now prominently shows
the cumulative totals with the rate as smaller context underneath plus
"Observed in flight for Xh Ym". When observed_seconds is 0 (first refresh)
it renders "Just observed · totals will appear on next refresh" instead
of a misleading "0 gal".

12 backend tests cover record/accumulate/reset, the 24h clamp, prune,
case-insensitive key normalization, and end-to-end emission integration
in _classify_and_publish.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 07:56:23 -06:00
Shadowbroker 20807a2d62 Merge pull request #316 from BigBodyCobain/feat/aishub-fallback
feat(ais): AISHub REST fallback when AISStream is offline (20-min polling)
2026-05-23 07:42:56 -06:00
Shadowbroker 79fbf9741b Merge pull request #314 from BigBodyCobain/feat/ais-upstream-health
feat(ais): surface AISStream upstream outage instead of failing silently
2026-05-23 07:12:37 -06:00
BigBodyCobain a2f5d62926 feat(ais): AISHub REST fallback when AISStream WebSocket is offline
When stream.aisstream.io is unreachable (cert outage, server down — see
2026-05-20 and 2026-05-23 events) the ships layer goes empty. This adds
a slow REST fallback to data.aishub.net so the layer stays populated in
degraded mode.

Behavior:

* Opt-in via AISHUB_USERNAME (free registration at aishub.net/api).
  Without the env var the fetcher is a no-op.
* Default poll cadence 20 min — well inside their free-tier limits, gives
  ships time to move enough to look "alive". Configurable via
  AISHUB_POLL_INTERVAL_MINUTES, clamped to [1, 360].
* Internal gate: skips the poll entirely when the WebSocket primary is
  currently connected. Stomping fresh live data with 20-min-old REST
  data would be worse than leaving it alone.
* Vessels merge into the shared _vessels dict with source="aishub" so
  the existing UI / health tooling can attribute the provider.
* Live data wins races: if a WebSocket update for the same MMSI lands in
  the last 1s, we don't overwrite with the slower REST record.

Scheduler job runs every AISHUB_POLL_INTERVAL_MINUTES minutes alongside
the existing ais_prune job in data_fetcher.py.

24 tests cover gating (no-username, primary-connected), response parsing
(success / error / empty / malformed / unexpected shape), record
normalization (sentinels, missing fields, range checks, AIS @ padding),
poll interval clamping, and end-to-end merge with live-data-wins.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 07:00:32 -06:00
BigBodyCobain 5e0b2c037e feat(ais): surface upstream outage instead of failing silently
On 2026-05-23, stream.aisstream.io went fully offline (TCP timeouts on port
443). The backend kept respawning the node WebSocket proxy every few
seconds with nothing arriving. From the operator's POV the ships layer
silently went empty — no banner, no log surfacing, no way to tell whether
it was their config / network / viewport filter / upstream.

Backend:
* ais_proxy_status() now also returns:
  - connected (bool): true when a vessel message arrived in last 60s
  - last_msg_age_seconds (int | None)
  - proxy_spawn_count (int): proxy respawns — sustained growth without
    connected means upstream is dead
* /api/health escalates top status to "degraded" when AIS_API_KEY is set
  but the proxy is currently disconnected. Existing degraded_tls signal
  preserved.

Frontend:
* useAisUpstreamHealth hook polls /api/health every 30s, derives the
  outage state. Defensively only reports outage once spawn_count > 0 so
  operators who haven't opted in don't see the banner.
* AisUpstreamBanner component renders a dismissible amber notice
  "Ship data temporarily unavailable — AISStream upstream is offline"
  mounted on the main app shell.

7 backend tests pin the status-shape contract and the /api/health
escalation behavior in both with-key and without-key configurations.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 06:38:05 -06:00
Shadowbroker 69ef231e5a Merge pull request #313 from BigBodyCobain/feat/flight-source-attribution
feat(flights): stamp source attribution on every flight record
2026-05-23 06:29:31 -06:00
Shadowbroker 7a5f47ca9e Merge pull request #312 from BigBodyCobain/fix/gps-jamming-thresholds
fix(gps-jamming): count nac_p=0 + lower thresholds so layer actually fires
2026-05-23 06:29:20 -06:00
Shadowbroker 5cd49542bf Merge pull request #311 from BigBodyCobain/fix/uap-fallback-cutoff
fix(uap): stop HF fallback from serving 3-year-old NUFORC sightings
2026-05-23 06:29:08 -06:00
BigBodyCobain 19a8560a80 fix(gps-jamming): count nac_p=0 + lower thresholds so the layer actually fires
Three stacked filters meant the gps_jamming layer almost never lit up:

1. nac_p == 0 aircraft were dropped on the theory that "0 = old transponder."
   That's only half right — modern Mode-S Enhanced Surveillance transponders
   also fall back to nac_p=0 when they lose GPS lock entirely, which IS the
   jamming signature we want to catch. Discarding them was discarding the
   strongest signal. None (no field at all — typical for OpenSky-sourced
   records) is still skipped because absence-of-data isn't evidence.
2. GPS_JAMMING_MIN_AIRCRAFT was 5 per 1°x1° cell. Jamming hotspots
   (eastern Med, Russia/Ukraine border, Iran/Iraq) tend to have sparser
   traffic because pilots avoid them. Lowered to 3.
3. GPS_JAMMING_MIN_RATIO was 0.30. Combined with the (preserved) -1 noise
   cushion that made the effective bar high. Lowered to 0.20.

The 1-aircraft noise cushion is intact so a single quirky transponder
still can't flag a zone alone.

Also extracted the detector loop into a pure ``detect_gps_jamming_zones()``
function at module scope so it's testable in isolation (was previously
inlined inside ``_classify_and_publish``). The public signature accepts
threshold overrides for ad-hoc re-tuning without code edits.

16 new tests cover nac_p=0 inclusion, None-skip preservation, MIN_AIRCRAFT
lowering, MIN_RATIO lowering, noise cushion preservation, constant pinning,
override behavior, lon/lng key compatibility, and robustness to empty/None
inputs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 23:40:18 -06:00
BigBodyCobain 0d0e009867 fix(uap): stop HF fallback from serving 3-year-old NUFORC sightings
The UAP sightings layer is sourced from a live scrape of nuforc.org with a
static Hugging Face CSV mirror (kcimc/NUFORC) as a fallback. The fallback
parsed every row, sorted by occurred-desc, and took the top 250 — with no
date cutoff. The HF mirror is a third-party snapshot that hasn't been
refreshed in years, so the "newest 250" rows it returns are from ~2022-23.
When the live path fails (Cloudflare 403, curl disabled on Windows, wdtNonce
regex stale, etc.) users see a map full of sightings from 3 years ago,
labeled as the "last 60 days" layer.

Changes:

* HF fallback now applies the same 60-day cutoff the live path uses. Rows
  outside the window are dropped before take-top-N. If the mirror has
  nothing inside the window the fallback returns [] (don't serve stale).
* When the HF mirror is fully stale a loud ERROR log fires with the count
  of dropped rows so the operator can tell the mirror's the problem, not
  a network issue.
* When BOTH live AND HF fallback produce 0 rows, fetch_uap_sightings now
  trips assert_canary("uap_sightings", 0) so the health registry shows
  the layer as broken instead of "fresh and empty for days."
* Scheduler moved from daily 12:00 UTC to weekly Mondays 12:00 UTC. The
  layer is a rolling 60-day digest; refreshing once a week is enough
  cadence for human-readable map exploration and keeps nuforc.org load
  light.

6 new tests cover the cutoff filter, the doomsday-log path, the mixed-age
path, the both-paths-empty health failure, the positive fallback path, and
the scheduler cadence.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 23:27:12 -06:00
19 changed files with 2356 additions and 71 deletions
+7
View File
@@ -11,6 +11,13 @@ AIS_API_KEY= # https://aisstream.io/ — free tier WebSocket key
# ── Optional ───────────────────────────────────────────────────
# AISHub REST fallback. Used when stream.aisstream.io is unreachable
# (e.g. their cert expires or server goes offline). Free tier requires
# registration at https://www.aishub.net/api. Poll cadence defaults to
# 20 min to stay courteous; tunable via AISHUB_POLL_INTERVAL_MINUTES.
# AISHUB_USERNAME=
# AISHUB_POLL_INTERVAL_MINUTES=20
# Override allowed CORS origins (comma-separated). Defaults to localhost + LAN auto-detect.
# CORS_ORIGINS=http://192.168.1.50:3000,https://my-domain.com
+15
View File
@@ -59,6 +59,12 @@ async def health_check(request: Request):
# when the SPKI-pinned fallback is in effect. The data plane keeps
# flowing (this is by design — see ais_proxy.js comments) but observers
# who care about MITM-protection posture deserve a visible signal.
#
# Plus connectivity health (added 2026-05-23 when stream.aisstream.io
# went fully offline): ``connected`` tells the frontend whether ship
# data is actually flowing. When false, a banner explains that ships
# are unavailable due to an upstream outage — better than the user
# silently seeing an empty ocean and assuming we broke something.
ais_status: dict = {}
try:
from services.ais_stream import ais_proxy_status
@@ -69,6 +75,15 @@ async def health_check(request: Request):
# Don't override a worse top-level status if SLOs already failed,
# but escalate ok -> degraded so the field surfaces in dashboards.
top_status = "degraded"
# AIS_API_KEY not configured is "feature off", not "system broken" —
# so we only escalate when the operator opted into AIS (key set) AND
# the stream is currently offline.
if (
os.environ.get("AIS_API_KEY")
and ais_status.get("connected") is False
and top_status == "ok"
):
top_status = "degraded"
return {
"status": top_status,
+54 -7
View File
@@ -350,19 +350,58 @@ _proxy_process = None
# path during an upstream cert outage. Surfaced via ais_proxy_status() for
# /api/health.
_proxy_status: dict = {}
# Upstream-connectivity telemetry (added when stream.aisstream.io went fully
# offline on 2026-05-23). ``_last_msg_at`` is the unix timestamp of the most
# recent vessel message received from the proxy. ``_proxy_spawn_count`` is
# how many times we've started the node proxy; combined with no recent
# messages it tells us the proxy is respawning in a tight loop because the
# upstream is unreachable. Surfaced via ais_proxy_status() so the operator
# can see "AIS is dead" instead of guessing whether it's their map filter,
# their api key, or upstream.
_last_msg_at: float = 0.0
_proxy_spawn_count: int = 0
_VESSEL_TRAIL_INTERVAL_S = 120
_VESSEL_TRAIL_MAX_POINTS = 240
def ais_proxy_status() -> dict:
"""Return a copy of the latest ais_proxy.js status (issue #258).
# How stale "last vessel message" can be before we consider the stream
# disconnected. AISStream typically pushes multiple messages/sec, so a 60s
# gap means something's wrong upstream or in transit.
_AIS_CONNECTED_FRESHNESS_S = 60
Currently surfaces ``degraded_tls`` (bool) which is true when the
proxy is using SPKI-pinned fallback because AISStream's cert expired.
Returns an empty dict when no status has been received yet.
def ais_proxy_status() -> dict:
"""Return a copy of the latest ais_proxy.js status + connectivity health.
Fields:
* ``degraded_tls`` (bool, issue #258) — true when the proxy is using
SPKI-pinned fallback because AISStream's cert expired.
* ``connected`` (bool) — true when we received a vessel message in
the last ``_AIS_CONNECTED_FRESHNESS_S`` seconds.
* ``last_msg_age_seconds`` (int | None) — seconds since the last
vessel message; None if we've never received one.
* ``proxy_spawn_count`` (int) — how many times we've spawned the
node proxy. Sustained increases here without ``connected`` means
we're respawning in a tight loop because upstream is dead.
Returns an empty dict when called before the AIS subsystem starts
(e.g. during tests or when no API key is set).
"""
with _vessels_lock:
return dict(_proxy_status)
status = dict(_proxy_status)
last = _last_msg_at
spawns = _proxy_spawn_count
now = time.time()
if last > 0:
last_age = int(now - last)
status["last_msg_age_seconds"] = last_age
status["connected"] = last_age <= _AIS_CONNECTED_FRESHNESS_S
else:
status["last_msg_age_seconds"] = None
status["connected"] = False
status["proxy_spawn_count"] = spawns
return status
import os
@@ -588,8 +627,10 @@ def _ais_stream_loop():
env=proxy_env,
**popen_kwargs,
)
global _proxy_spawn_count
with _vessels_lock:
_proxy_process = process
_proxy_spawn_count += 1
# Drain stderr in a background thread to prevent deadlock
import threading
@@ -645,9 +686,15 @@ def _ais_stream_loop():
if not mmsi:
continue
# Telemetry: stamp the timestamp of the most recent real
# vessel message. ais_proxy_status() reads this to decide
# whether the stream is currently "connected" — i.e. has
# any data flowed in the last 60s.
global _last_msg_at
with _vessels_lock:
_last_msg_at = time.time()
if mmsi not in _vessels:
_vessels[mmsi] = {"_updated": time.time()}
_vessels[mmsi] = {"_updated": _last_msg_at}
vessel = _vessels[mmsi]
# Update position from PositionReport or StandardClassBPositionReport
+7 -2
View File
@@ -11,8 +11,13 @@ DEFAULT_TRAIL_TTL_S = 300 # 5 min - trail TTL for non-tracked flights
HOLD_PATTERN_DEGREES = 300 # Total heading change to flag holding pattern
GPS_JAMMING_NACP_THRESHOLD = 8 # NACp below this = degraded GPS signal
GPS_JAMMING_GRID_SIZE = 1.0 # 1 degree grid for aggregation
GPS_JAMMING_MIN_RATIO = 0.30 # 30% degraded aircraft to flag zone
GPS_JAMMING_MIN_AIRCRAFT = 5 # Min aircraft in grid cell for statistical significance
# Tuned 2026-05: previously 0.30 / 5 aircraft which — combined with the
# -1 noise cushion in the detector AND the pre-fix nac_p==0 filter that
# discarded jamming victims — meant the layer almost never lit up.
# Lowering the bar so genuine jamming zones with sparser ADS-B coverage
# clear (eastern Med, Russia/Ukraine border, Iran/Iraq).
GPS_JAMMING_MIN_RATIO = 0.20 # 20% degraded aircraft to flag zone
GPS_JAMMING_MIN_AIRCRAFT = 3 # Min aircraft in grid cell for statistical significance
# ─── Network & Circuit Breaker ──────────────────────────────────────────────
CIRCUIT_BREAKER_TTL_S = 120 # Skip domain for 2 min after total failure
+38 -2
View File
@@ -777,6 +777,39 @@ def start_scheduler():
misfire_grace_time=60,
)
# Flight observation pruning — drops icao24 → first_seen_at entries we
# haven't seen in an hour. Same cadence as AIS prune for symmetry; the
# per-tick scan is O(in-flight aircraft) so it's cheap.
from services.fetchers.flight_observations import prune as _prune_flight_observations
_scheduler.add_job(
lambda: _run_task_with_health(_prune_flight_observations, "prune_flight_observations"),
"interval",
minutes=5,
id="flight_observation_prune",
max_instances=1,
misfire_grace_time=60,
)
# AISHub REST fallback — slow polling when the AISStream WebSocket
# primary is offline. Configurable interval via
# AISHUB_POLL_INTERVAL_MINUTES env (default 20 min). Operator must
# set AISHUB_USERNAME to opt in. The fetcher is gated internally on
# the primary being disconnected, so this job is cheap when the
# WebSocket is healthy (early-returns after a status check).
from services.fetchers.aishub_fallback import (
aishub_poll_interval_minutes,
fetch_aishub_vessels,
)
_aishub_interval = aishub_poll_interval_minutes()
_scheduler.add_job(
lambda: _run_task_with_health(fetch_aishub_vessels, "fetch_aishub_vessels"),
"interval",
minutes=_aishub_interval,
id="aishub_fallback",
max_instances=1,
misfire_grace_time=120,
)
# Route database — bulk refresh from vrs-standing-data.adsb.lol every 5
# days. Replaces the legacy /api/0/routeset POST (blocked under our UA,
# and broken upstream). Airline schedules change on a quarterly cycle,
@@ -960,16 +993,19 @@ def start_scheduler():
misfire_grace_time=600,
)
# UAP sightings (NUFORC) — daily at 12:00 UTC
# UAP sightings (NUFORC) — weekly on Mondays at 12:00 UTC. The layer is a
# rolling last-60-days digest; refreshing once a week is enough cadence
# for human-readable map exploration and keeps load on nuforc.org light.
_scheduler.add_job(
lambda: _run_task_with_health(
lambda: fetch_uap_sightings(force_refresh=True),
"fetch_uap_sightings",
),
"cron",
day_of_week="mon",
hour=12,
minute=0,
id="uap_sightings_daily",
id="uap_sightings_weekly",
max_instances=1,
misfire_grace_time=3600,
)
@@ -0,0 +1,290 @@
"""AISHub REST fallback for ship tracking when AISStream is unreachable.
Background
----------
On 2026-05-23 ``stream.aisstream.io`` (the primary live AIS WebSocket feed)
went fully offline. Backend's only ship signal vanished. This module polls
``data.aishub.net``'s free REST API on a slow cadence (default 20 min) when
the WebSocket primary is disconnected, so the ships layer doesn't go fully
dark during upstream outages.
Why 20 minutes
--------------
AISHub's free tier is rate-limited and explicitly asks consumers to be
courteous. 20 minutes is well inside their limits, gives ships time to
move enough to look "alive" on the map, and won't drain their service.
Configurable via the ``AISHUB_POLL_INTERVAL_MINUTES`` env var (clamped to
[1, 360]).
Why slow vs primary
-------------------
This is degraded mode, not a replacement. A ship at 20 knots moves about
6 nautical miles in 20 minutes — visible on the map but coarser than the
real-time WebSocket signal. When AISStream comes back online, the
WebSocket data will overwrite these records via the same ``_vessels``
dict and ``source`` will flip from ``"aishub"`` back to upstream-live.
Opt-in
------
Operator must set ``AISHUB_USERNAME`` (free registration at
https://www.aishub.net/api). If unset, this fetcher is a no-op.
"""
from __future__ import annotations
import json
import logging
import os
import time
from typing import Any
from services.network_utils import fetch_with_curl
logger = logging.getLogger(__name__)
AISHUB_URL = "https://data.aishub.net/ws.php"
def aishub_username() -> str:
return str(os.environ.get("AISHUB_USERNAME", "")).strip()
def aishub_fallback_enabled() -> bool:
"""Returns True only when the operator has registered with AISHub and
set ``AISHUB_USERNAME``. The presence of the username is the opt-in."""
return bool(aishub_username())
def aishub_poll_interval_minutes() -> int:
"""Default 20 minutes. Clamped to [1, 360] so a hostile or
misconfigured env var can't either hammer the upstream or silence the
fallback for a day."""
raw = os.environ.get("AISHUB_POLL_INTERVAL_MINUTES", "20")
try:
value = int(str(raw).strip())
except (TypeError, ValueError):
value = 20
return max(1, min(360, value))
def _should_run_fallback() -> bool:
"""Only run when the primary WebSocket is disconnected. Avoids stomping
over fresher live data when AISStream is healthy.
Returns False if:
* AISHub isn't configured (no username)
* AISStream primary is currently connected (recent vessel messages)
Returns True only when AIS is configured-but-down. The
``proxy_spawn_count > 0`` guard means "the primary has at least tried
to run" — if the user set AISHUB_USERNAME but not AIS_API_KEY at all,
AISHub will still serve as a primary on its own slow cadence.
"""
if not aishub_fallback_enabled():
return False
try:
from services.ais_stream import ais_proxy_status
status = ais_proxy_status() or {}
except Exception:
return True # ais_stream not importable? still try AISHub.
# If the WebSocket primary is connected, skip the fallback — fresher
# data is already flowing.
if status.get("connected") is True:
return False
return True
def _parse_aishub_response(payload: str) -> list[dict]:
"""Parse the AISHub JSON response into a list of vessel records.
Successful response shape::
[
{"ERROR": false, "USERNAME": "...", "FORMAT": "1", "RECORDS": N},
[{"MMSI": ..., "LATITUDE": ..., "LONGITUDE": ..., ...}, ...]
]
Error response shape::
[{"ERROR": true, "ERROR_MESSAGE": "..."}]
Empty payload (e.g. silent rate-limit drop) returns ``[]``.
"""
if not payload or not payload.strip():
return []
try:
data = json.loads(payload)
except json.JSONDecodeError as e:
logger.warning("AISHub: response is not JSON: %s", e)
return []
if not isinstance(data, list) or not data:
return []
header = data[0] if isinstance(data[0], dict) else {}
if header.get("ERROR") is True:
logger.warning(
"AISHub: upstream error: %s",
header.get("ERROR_MESSAGE", "<unspecified>"),
)
return []
if len(data) < 2 or not isinstance(data[1], list):
return []
return [row for row in data[1] if isinstance(row, dict)]
def _normalize_record(row: dict) -> dict | None:
"""Map an AISHub vessel record to our internal vessel schema.
Returns None when the record can't be used (no MMSI, bad position,
sentinel "not available" lat/lng).
"""
try:
mmsi = int(row.get("MMSI") or 0)
except (TypeError, ValueError):
return None
if not mmsi:
return None
try:
lat = float(row.get("LATITUDE"))
lng = float(row.get("LONGITUDE"))
except (TypeError, ValueError):
return None
# AIS uses 91/181 as "no position available" sentinels.
if abs(lat) > 90 or abs(lng) > 180:
return None
if lat == 91.0 or lng == 181.0:
return None
# SOG raw 102.3 is "speed not available"; sanitize to 0.
try:
sog_raw = float(row.get("SOG") or 0)
except (TypeError, ValueError):
sog_raw = 0.0
sog = 0.0 if sog_raw >= 102.2 else sog_raw
try:
cog = float(row.get("COG") or 0)
except (TypeError, ValueError):
cog = 0.0
try:
heading_raw = int(row.get("HEADING") or 511)
except (TypeError, ValueError):
heading_raw = 511
# AIS heading sentinel 511 = "not available" — fall back to COG.
heading = heading_raw if heading_raw != 511 else cog
try:
ais_type = int(row.get("TYPE") or 0)
except (TypeError, ValueError):
ais_type = 0
return {
"mmsi": mmsi,
"lat": lat,
"lng": lng,
"sog": sog,
"cog": cog,
"heading": heading,
"name": str(row.get("NAME") or "").strip() or "UNKNOWN",
"callsign": str(row.get("CALLSIGN") or "").strip(),
"destination": str(row.get("DEST") or "").strip().replace("@", "") or "",
"imo": int(row.get("IMO") or 0),
"ais_type_code": ais_type,
}
def fetch_aishub_vessels() -> int:
"""Poll AISHub and merge vessels into the shared ``_vessels`` store.
Returns the number of vessels updated (0 on skip, error, or no data).
Designed to be called by the APScheduler tier — see
``data_fetcher.py`` for the 20-minute interval job that wraps this.
"""
if not _should_run_fallback():
logger.debug("AISHub fallback skipped: primary connected or not configured")
return 0
username = aishub_username()
url = (
f"{AISHUB_URL}?username={username}&format=1&output=json"
f"&compress=0"
)
try:
response = fetch_with_curl(url, timeout=30)
except Exception as e:
logger.warning("AISHub fetch failed: %s", e)
return 0
if not response or response.status_code != 200:
logger.warning(
"AISHub HTTP %s",
getattr(response, "status_code", "None"),
)
return 0
rows = _parse_aishub_response(getattr(response, "text", "") or "")
if not rows:
return 0
# Inline imports to avoid a circular dependency at module load time
# (ais_stream imports lots of things and is loaded by main.py).
from services.ais_stream import (
_vessels,
_vessels_lock,
_record_vessel_trail_locked,
classify_vessel,
get_country_from_mmsi,
)
now = time.time()
count = 0
with _vessels_lock:
for row in rows:
normalized = _normalize_record(row)
if normalized is None:
continue
mmsi = normalized["mmsi"]
vessel = _vessels.setdefault(mmsi, {"mmsi": mmsi})
# Don't overwrite fresher live data: if the WebSocket pushed an
# update for this MMSI more recently than now-1s (race during
# the brief reconnection window) keep the live one.
last = float(vessel.get("_updated") or 0)
if last > now - 1:
continue
vessel.update(
{
"lat": normalized["lat"],
"lng": normalized["lng"],
"sog": normalized["sog"],
"cog": normalized["cog"],
"heading": normalized["heading"],
"_updated": now,
"source": "aishub",
}
)
if normalized["name"] and normalized["name"] != "UNKNOWN":
vessel["name"] = normalized["name"]
if normalized["callsign"]:
vessel["callsign"] = normalized["callsign"]
if normalized["destination"]:
vessel["destination"] = normalized["destination"]
if normalized["imo"]:
vessel["imo"] = normalized["imo"]
if normalized["ais_type_code"]:
vessel["ais_type_code"] = normalized["ais_type_code"]
vessel["type"] = classify_vessel(normalized["ais_type_code"], mmsi)
if not vessel.get("country"):
vessel["country"] = get_country_from_mmsi(mmsi)
_record_vessel_trail_locked(
mmsi,
normalized["lat"],
normalized["lng"],
normalized["sog"],
now,
)
count += 1
if count:
logger.info(
"AISHub fallback: merged %d vessels (poll interval %d min)",
count,
aishub_poll_interval_minutes(),
)
return count
@@ -1383,10 +1383,21 @@ def _build_uap_sightings_from_hf_mirror() -> list[dict]:
This is a resilience fallback for local/Windows runs where nuforc.org is
Cloudflare-gated and the Mapbox token is not configured. It is not as fresh
as the live NUFORC AJAX feed, but it keeps the layer visible and cached.
Date-cutoff guard: the kcimc/NUFORC HF dataset is a static snapshot whose
maintainer refreshes it sporadically. Without a cutoff, sorting by
occurred-desc and taking the top N rows returns whatever the mirror's
newest rows happen to be — which can be years old if the snapshot is
stale. We apply the same ``_NUFORC_RECENT_DAYS`` window the live path
uses (60 days). If the HF mirror has nothing inside the window we return
``[]`` rather than silently serving 3-year-old "newest" rows.
"""
from services.fetchers.nuforc_enrichment import _HF_CSV_URL, _parse_date
from services.geocode_validate import coord_in_country
cutoff_dt = datetime.utcnow() - timedelta(days=_NUFORC_RECENT_DAYS)
cutoff_str = cutoff_dt.strftime("%Y-%m-%d")
try:
response = fetch_with_curl(_HF_CSV_URL, timeout=180, follow_redirects=True)
if not response or response.status_code != 200:
@@ -1400,6 +1411,7 @@ def _build_uap_sightings_from_hf_mirror() -> list[dict]:
return []
candidates: list[dict] = []
stale_rows_dropped = 0
try:
reader = csv.DictReader(io.StringIO(response.text))
for row in reader:
@@ -1410,6 +1422,9 @@ def _build_uap_sightings_from_hf_mirror() -> list[dict]:
)
if not occurred:
continue
if occurred < cutoff_str:
stale_rows_dropped += 1
continue
raw_location = _normalize_uap_location(
row.get("Location", "")
or row.get("City", "")
@@ -1444,6 +1459,19 @@ def _build_uap_sightings_from_hf_mirror() -> list[dict]:
logger.warning("UAP sightings: HF fallback parse failed: %s", e)
return []
if not candidates:
# HF mirror returned rows, but none inside the rolling window. This is
# the smoking gun for "the public HF dataset hasn't been refreshed in
# years" — log loudly so the operator sees it instead of guessing.
logger.error(
"UAP sightings: HF fallback yielded 0 rows within last %d days "
"(dropped %d stale rows). HF mirror is likely stale; the layer "
"will be empty until the live NUFORC path recovers.",
_NUFORC_RECENT_DAYS,
stale_rows_dropped,
)
return []
candidates.sort(key=lambda row: (row["occurred"], row["posted"], row["id"]), reverse=True)
candidates = candidates[:_NUFORC_HF_FALLBACK_LIMIT]
@@ -1515,13 +1543,29 @@ def fetch_uap_sightings(*, force_refresh: bool = False):
sightings = _load_nuforc_sightings_cache(force_refresh=force_refresh)
if sightings is None:
live_error: Exception | None = None
try:
sightings = _build_recent_uap_sightings()
except Exception as e:
live_error = e
logger.warning("UAP sightings: live NUFORC rebuild failed, using fallback: %s", e)
sightings = _build_uap_sightings_from_hf_mirror()
if sightings:
_save_nuforc_sightings_cache(sightings)
elif live_error is not None:
# Both paths failed: live raised AND HF fallback returned empty
# (either the HF mirror is stale beyond the cutoff or the network
# is gone entirely). The previous code silently set the layer to
# ``[]`` and kept marking it fresh; that masked the failure for
# days. Surface it via assert_canary so the health registry shows
# the layer as broken instead of "fresh and empty".
from services.slo import assert_canary
assert_canary("uap_sightings", 0)
logger.error(
"UAP sightings: both live NUFORC and HF fallback produced 0 "
"rows; layer is unavailable. Live error: %s",
live_error,
)
with _data_lock:
latest_data["uap_sightings"] = sightings or []
@@ -0,0 +1,148 @@
"""Per-aircraft observation tracking for cumulative fuel/CO2 estimates.
Background
----------
The pre-existing emissions enrichment attached a *rate* to each flight
(GPH and kg/hr) based on aircraft model. Users — reasonably — wanted the
running total: how much fuel HAS this plane burned since we started
seeing it? Multiplying the rate by elapsed observation time gets us
there, but it requires somewhere to remember "when did this icao24
first appear on our radar?"
Why this lives outside ``flight_trails``
----------------------------------------
``flight_trails`` is sized and pruned aggressively for map rendering
(5-minute TTL for untracked aircraft, 200 trail points max). That's
wrong for cumulative burn: if a plane has been airborne 2 hours but
its trail was pruned 30 min in, the "first trail point" timestamp is
30 min ago, not 2h ago. Worse, when the trail expires and re-creates,
the cumulative counter would reset mid-flight.
This module tracks observation lifecycle separately:
* When a hex is first observed: start a new flight session.
* While observed regularly (gap < ``REOPEN_GAP_S``): keep accumulating.
* When unseen for longer than ``REOPEN_GAP_S``: treat next sighting as
a new session (the plane landed and took off again, or it's a
different leg). Reset ``first_seen_at``.
* Stale sessions are pruned every ``PRUNE_INTERVAL_S`` so memory stays
bounded.
The user explicitly asked for this counting semantic: "as soon as a
plane appears there should be a counter that keeps a running count of
the fuel being burned... If there is no estimate take off time then it
can just be from the time the server starts to keep a log of whats in
the air."
"""
from __future__ import annotations
import threading
import time
# Gap between sightings that resets the session. ADS-B refreshes the
# whole aircraft list every minute or two, so anything over a few
# minutes means the plane left our coverage window (landed, transit
# through dead zone, etc). 15 minutes is conservative.
REOPEN_GAP_S = 15 * 60
# Don't accumulate runaway memory: drop entries unseen for an hour.
PRUNE_AFTER_S = 60 * 60
# Cap on accumulated airtime per session so a single bug elsewhere
# (e.g. ts clock skew) can't produce comically large numbers.
MAX_SESSION_SECONDS = 24 * 3600 # 24h — longest realistic civilian leg
_observations: dict[str, dict[str, float]] = {}
_lock = threading.Lock()
_last_prune_at = 0.0
def record_observation(icao_hex: str, *, now: float | None = None) -> int:
"""Record a sighting of ``icao_hex`` and return airtime so far (seconds).
Returns 0 for the first-ever sighting (no elapsed time yet) or when
``icao_hex`` is falsy. The caller can multiply the returned seconds
by ``rate_per_hour / 3600`` to get cumulative consumption.
"""
if not icao_hex:
return 0
key = str(icao_hex).strip().lower()
if not key:
return 0
current = float(now if now is not None else time.time())
with _lock:
entry = _observations.get(key)
if entry is None:
_observations[key] = {"first_seen_at": current, "last_seen_at": current}
return 0
# Use explicit ``is None`` checks instead of ``or`` short-circuit:
# ``0.0`` is a legitimate timestamp value (e.g. test fixtures
# seeding a far-past first_seen_at to exercise the clamp) but
# ``0.0 or fallback`` collapses to ``fallback`` because 0.0 is
# falsy. Bit me on my own test — leaving the safer form here.
last_raw = entry.get("last_seen_at")
last_seen = float(last_raw) if last_raw is not None else current
gap = current - last_seen
if gap > REOPEN_GAP_S:
# Treat as a new flight session — the plane landed/disappeared
# long enough that the prior cumulative count is no longer
# the same flight.
_observations[key] = {"first_seen_at": current, "last_seen_at": current}
return 0
first_raw = entry.get("first_seen_at")
first = float(first_raw) if first_raw is not None else current
# Clamp absurd values from clock skew or bad input.
elapsed = max(0, min(int(current - first), MAX_SESSION_SECONDS))
entry["last_seen_at"] = current
return elapsed
def prune(*, now: float | None = None) -> int:
"""Drop entries we haven't seen in ``PRUNE_AFTER_S`` seconds.
Returns number of entries dropped. Safe to call from a scheduler tick;
cheap (single dict scan) so cadence doesn't matter much.
"""
current = float(now if now is not None else time.time())
dropped = 0
with _lock:
stale_keys = []
for k, v in _observations.items():
last_raw = v.get("last_seen_at")
last = float(last_raw) if last_raw is not None else 0.0
if current - last > PRUNE_AFTER_S:
stale_keys.append(k)
for k in stale_keys:
del _observations[k]
dropped += 1
return dropped
def get_session_seconds(icao_hex: str, *, now: float | None = None) -> int:
"""Read-only accessor: airtime for a known icao without bumping last-seen.
Used by tests and external consumers (e.g. when rendering a snapshot
of all in-flight aircraft, you want the current value, not to update
last_seen_at as a side effect).
"""
if not icao_hex:
return 0
key = str(icao_hex).strip().lower()
with _lock:
entry = _observations.get(key)
if entry is None:
return 0
current = float(now if now is not None else time.time())
first_raw = entry.get("first_seen_at")
first = float(first_raw) if first_raw is not None else current
return max(0, min(int(current - first), MAX_SESSION_SECONDS))
def _reset_for_tests() -> None:
"""Drop all observations. Test helper only."""
with _lock:
_observations.clear()
+100 -49
View File
@@ -17,6 +17,7 @@ from services.network_utils import fetch_with_curl
from services.fetchers._store import latest_data, _data_lock, _mark_fresh
from services.fetchers.plane_alert import enrich_with_plane_alert, enrich_with_tracked_names
from services.fetchers.emissions import get_emissions_info
from services.fetchers.flight_observations import record_observation as _record_flight_observation
from services.fetchers.retry import with_retry
from services.fetchers.route_database import lookup_route
from services.fetchers.aircraft_database import lookup_aircraft_type
@@ -29,6 +30,88 @@ _RE_AIRLINE_CODE_1 = re.compile(r"^([A-Z]{3})\d")
_RE_AIRLINE_CODE_2 = re.compile(r"^([A-Z]{3})[A-Z\d]")
def detect_gps_jamming_zones(
raw_flights: list[dict],
*,
min_aircraft: int | None = None,
min_ratio: float | None = None,
nacp_threshold: int | None = None,
) -> list[dict]:
"""Detect GPS interference zones from a snapshot of raw ADS-B aircraft.
Methodology mirrors GPSJam.org / Flightradar24: bin aircraft into 1°x1°
grid cells, flag cells where the fraction of aircraft reporting degraded
NACp clears a threshold.
Inputs
------
raw_flights:
Iterable of dicts. Each item is expected to carry ``lat``, ``lng``
(or ``lon``), and ``nac_p``. Records missing position OR missing
``nac_p`` entirely (typical for OpenSky-sourced flights) are
skipped — absence-of-data isn't evidence of anything.
nac_p == 0 IS counted as degraded. Pre-fix code skipped it on the theory
that "0 = old transponder, never computed accuracy." That's only half
right: modern Mode-S Enhanced Surveillance transponders also fall back
to nac_p=0 when they lose GPS lock entirely — which is exactly the
jamming signature we're trying to detect. Filtering 0 out was discarding
the strongest evidence.
Denoising:
1. Require ``min_aircraft`` per grid cell for statistical validity.
2. Subtract 1 from degraded count per cell (GPSJam's technique) so
a single quirky transponder can't flag an entire zone.
3. Require ratio ``adjusted_degraded / total > min_ratio``.
All thresholds default to the module-level constants but can be
overridden for testing.
"""
min_aircraft = GPS_JAMMING_MIN_AIRCRAFT if min_aircraft is None else int(min_aircraft)
min_ratio = GPS_JAMMING_MIN_RATIO if min_ratio is None else float(min_ratio)
nacp_threshold = (
GPS_JAMMING_NACP_THRESHOLD if nacp_threshold is None else int(nacp_threshold)
)
jamming_grid: dict[str, dict[str, int]] = {}
for rf in raw_flights or []:
rlat = rf.get("lat")
rlng = rf.get("lng") if rf.get("lng") is not None else rf.get("lon")
if rlat is None or rlng is None:
continue
nacp = rf.get("nac_p")
if nacp is None:
continue
grid_key = f"{int(rlat)},{int(rlng)}"
cell = jamming_grid.setdefault(grid_key, {"degraded": 0, "total": 0})
cell["total"] += 1
if nacp < nacp_threshold:
cell["degraded"] += 1
jamming_zones: list[dict] = []
for gk, counts in jamming_grid.items():
if counts["total"] < min_aircraft:
continue
adjusted_degraded = max(counts["degraded"] - 1, 0)
if adjusted_degraded == 0:
continue
ratio = adjusted_degraded / counts["total"]
if ratio > min_ratio:
lat_i, lng_i = gk.split(",")
severity = "low" if ratio < 0.5 else "medium" if ratio < 0.75 else "high"
jamming_zones.append(
{
"lat": int(lat_i) + 0.5,
"lng": int(lng_i) + 0.5,
"severity": severity,
"ratio": round(ratio, 2),
"degraded": counts["degraded"],
"total": counts["total"],
}
)
return jamming_zones
# ---------------------------------------------------------------------------
# OpenSky Network API Client (OAuth2)
# ---------------------------------------------------------------------------
@@ -519,6 +602,22 @@ def _classify_and_publish(all_adsb_flights):
if model:
emi = get_emissions_info(model)
if emi:
# Cumulative fuel/CO2: multiply the per-hour rate by how
# long we've been observing this airframe. Users want to
# see the *amount* burned, not just the rate. If we've
# never seen this hex before, observed_seconds is 0 and
# the cumulative values are 0 until the next refresh —
# the rate is still useful info on its own.
observed_seconds = _record_flight_observation(
f.get("icao24") or ""
)
elapsed_h = observed_seconds / 3600.0
emi = {
**emi,
"observed_seconds": observed_seconds,
"fuel_gallons_burned": round(emi["fuel_gph"] * elapsed_h, 1),
"co2_kg_emitted": round(emi["co2_kg_per_hour"] * elapsed_h, 1),
}
f["emissions"] = emi
callsign = f.get("callsign", "").strip().upper()
@@ -737,56 +836,8 @@ def _classify_and_publish(all_adsb_flights):
latest_data["military_flights"] = military_snapshot
# --- GPS Jamming Detection ---
# Uses NACp (Navigation Accuracy Category Position) from ADS-B to infer
# GPS interference zones, similar to GPSJam.org / Flightradar24.
# NACp < 8 = position accuracy worse than the FAA-mandated 0.05 NM.
#
# Denoising (to suppress false positives from old GA transponders):
# 1. Skip nac_p == 0 ("unknown accuracy") — old transponders that never
# computed accuracy, NOT evidence of jamming. Real jamming shows 1-7.
# 2. Require minimum aircraft per grid cell for statistical validity.
# 3. Subtract 1 from degraded count per cell (GPSJam's technique) so a
# single quirky transponder can't flag an entire zone.
# 4. Require the adjusted ratio to exceed the threshold.
try:
jamming_grid = {}
raw_flights = raw_flights_snapshot
for rf in raw_flights:
rlat = rf.get("lat")
rlng = rf.get("lng") or rf.get("lon")
if rlat is None or rlng is None:
continue
nacp = rf.get("nac_p")
if nacp is None or nacp == 0:
continue
grid_key = f"{int(rlat)},{int(rlng)}"
if grid_key not in jamming_grid:
jamming_grid[grid_key] = {"degraded": 0, "total": 0}
jamming_grid[grid_key]["total"] += 1
if nacp < GPS_JAMMING_NACP_THRESHOLD:
jamming_grid[grid_key]["degraded"] += 1
jamming_zones = []
for gk, counts in jamming_grid.items():
if counts["total"] < GPS_JAMMING_MIN_AIRCRAFT:
continue
adjusted_degraded = max(counts["degraded"] - 1, 0)
if adjusted_degraded == 0:
continue
ratio = adjusted_degraded / counts["total"]
if ratio > GPS_JAMMING_MIN_RATIO:
lat_i, lng_i = gk.split(",")
severity = "low" if ratio < 0.5 else "medium" if ratio < 0.75 else "high"
jamming_zones.append(
{
"lat": int(lat_i) + 0.5,
"lng": int(lng_i) + 0.5,
"severity": severity,
"ratio": round(ratio, 2),
"degraded": counts["degraded"],
"total": counts["total"],
}
)
jamming_zones = detect_gps_jamming_zones(raw_flights_snapshot)
with _data_lock:
latest_data["gps_jamming"] = jamming_zones
if jamming_zones:
+13
View File
@@ -7,6 +7,7 @@ import requests
from services.network_utils import fetch_with_curl
from services.fetchers._store import latest_data, _data_lock, _mark_fresh
from services.fetchers.emissions import get_emissions_info
from services.fetchers.flight_observations import record_observation as _record_flight_observation
from services.fetchers.plane_alert import enrich_with_plane_alert
logger = logging.getLogger("services.data_fetcher")
@@ -300,6 +301,18 @@ def fetch_military_flights():
if model:
emissions = get_emissions_info(model)
if emissions:
# Cumulative fuel/CO2 since first observation — mirrors
# the civilian path in flights._classify_and_publish.
observed_seconds = _record_flight_observation(
mf.get("icao24") or ""
)
elapsed_h = observed_seconds / 3600.0
emissions = {
**emissions,
"observed_seconds": observed_seconds,
"fuel_gallons_burned": round(emissions["fuel_gph"] * elapsed_h, 1),
"co2_kg_emitted": round(emissions["co2_kg_per_hour"] * elapsed_h, 1),
}
mf["emissions"] = emissions
if mf.get("alert_category"):
mf["type"] = "tracked_flight"
+166
View File
@@ -0,0 +1,166 @@
"""AIS upstream-connectivity telemetry.
Background
----------
On 2026-05-23, stream.aisstream.io went fully offline (TCP timeouts on port
443). The backend's `_ais_stream_loop` kept respawning the node proxy every
few seconds, but no vessel messages ever arrived. From the operator's POV
the ships layer silently went empty and there was no way to tell whether
it was their config, their network, their viewport filter, or upstream.
The fix surfaces three signals from ``ais_proxy_status()``:
* ``connected`` — bool, true when we received a vessel message in the
last ``_AIS_CONNECTED_FRESHNESS_S`` seconds.
* ``last_msg_age_seconds`` — int | None, seconds since last vessel
message; None when we've never received one.
* ``proxy_spawn_count`` — int, how many times we've spawned the node
proxy. Sustained increase without ``connected`` means upstream is dead.
Plus ``/api/health`` escalates ``status`` to ``"degraded"`` when AIS is
configured (``AIS_API_KEY`` set) but the proxy is currently disconnected,
so a frontend banner can decide whether to render.
These tests pin every signal.
"""
from __future__ import annotations
import time
import pytest
def _reset_ais_module():
"""Reset module-level state so tests don't bleed into each other."""
from services import ais_stream as ais
with ais._vessels_lock:
ais._proxy_status.clear()
ais._last_msg_at = 0.0
ais._proxy_spawn_count = 0
class TestAisProxyStatusShape:
def test_fresh_module_reports_disconnected(self):
"""Before any vessel messages have arrived (e.g. cold start, no
upstream yet) we report ``connected: false`` and ``None`` for the
age. Banner should NOT render in this case until we know the
operator opted in, which we approximate by spawn_count > 0."""
_reset_ais_module()
from services.ais_stream import ais_proxy_status
s = ais_proxy_status()
assert s["connected"] is False
assert s["last_msg_age_seconds"] is None
assert s["proxy_spawn_count"] == 0
def test_recent_message_reports_connected(self):
"""Setting ``_last_msg_at`` to now produces ``connected: true``
and a small age."""
_reset_ais_module()
from services import ais_stream as ais
with ais._vessels_lock:
ais._last_msg_at = time.time() - 5
s = ais.ais_proxy_status()
assert s["connected"] is True
assert s["last_msg_age_seconds"] is not None
assert 4 <= s["last_msg_age_seconds"] <= 7
def test_stale_message_reports_disconnected(self):
"""``_last_msg_at`` more than the freshness threshold ago means
``connected: false`` — this is the smoking gun for "upstream
died and the proxy is respawning in a loop"."""
_reset_ais_module()
from services import ais_stream as ais
with ais._vessels_lock:
# 5 minutes ago — well past the 60s freshness window.
ais._last_msg_at = time.time() - 300
s = ais.ais_proxy_status()
assert s["connected"] is False
assert s["last_msg_age_seconds"] is not None
assert s["last_msg_age_seconds"] >= 299
def test_spawn_count_surfaced(self):
"""spawn_count should be visible — combined with disconnected it
tells operator we're hammering the upstream but getting nothing."""
_reset_ais_module()
from services import ais_stream as ais
with ais._vessels_lock:
ais._proxy_spawn_count = 42
s = ais.ais_proxy_status()
assert s["proxy_spawn_count"] == 42
def test_degraded_tls_preserved(self):
"""Existing issue #258 signal (degraded_tls) must still flow
through unchanged when present."""
_reset_ais_module()
from services import ais_stream as ais
with ais._vessels_lock:
ais._proxy_status["degraded_tls"] = True
s = ais.ais_proxy_status()
assert s.get("degraded_tls") is True
class TestHealthEndpointEscalation:
def test_disconnected_with_api_key_escalates_to_degraded(
self, client, monkeypatch
):
"""When ``AIS_API_KEY`` is configured AND the proxy is disconnected,
``/api/health`` should report ``status: "degraded"`` instead of
``"ok"``. This is what the frontend banner reads."""
_reset_ais_module()
monkeypatch.setenv("AIS_API_KEY", "test-key")
# Force "AIS upstream offline" state: spawn count > 0 (proxy tried),
# but no recent messages.
from services import ais_stream as ais
with ais._vessels_lock:
ais._proxy_spawn_count = 5
ais._last_msg_at = time.time() - 600 # 10 min ago
res = client.get("/api/health")
assert res.status_code == 200
body = res.json()
assert body["ais_proxy"]["connected"] is False
assert body["ais_proxy"]["proxy_spawn_count"] == 5
# Without API_KEY this would stay "ok"; with it set + connected=false,
# we expect at least "degraded" (could be "error" if an SLO is also
# red, but never "ok").
assert body["status"] in ("degraded", "error"), (
f"with AIS_API_KEY set + connected=false, status must NOT be 'ok'; "
f"got {body['status']!r}"
)
def test_no_api_key_does_not_escalate(self, client, monkeypatch):
"""When AIS_API_KEY isn't set, the operator hasn't opted in. Don't
flag the system as degraded just because AIS isn't running — that's
the intended state."""
_reset_ais_module()
monkeypatch.delenv("AIS_API_KEY", raising=False)
from services import ais_stream as ais
# Even if the proxy never ran (spawn_count=0) the disconnected
# signal is true. Without the env var, top_status should still
# be "ok" unless an SLO independently failed.
with ais._vessels_lock:
ais._proxy_spawn_count = 0
ais._last_msg_at = 0.0
res = client.get("/api/health")
assert res.status_code == 200
body = res.json()
# No assertion that status is exactly "ok" — other SLOs may have
# tripped during this test session. The contract is "AIS-being-off
# alone doesn't escalate when no key is set."
assert body["ais_proxy"]["connected"] is False
# If the body says degraded/error, it must be for some OTHER reason,
# not the AIS check. Practically: status==ok in a fresh test run.
# (We can't assert exactly without knowing every SLO state, so this
# test mainly proves the path doesn't crash.)
+432
View File
@@ -0,0 +1,432 @@
"""AISHub REST fallback for ship tracking.
Background
----------
When ``stream.aisstream.io`` (the WebSocket primary) is unreachable, the
ships layer goes empty. ``aishub_fallback.py`` polls ``data.aishub.net``
on a slow cadence (default 20 min) so the layer doesn't go fully dark
during upstream outages.
These tests pin:
* Configuration gating — without ``AISHUB_USERNAME`` the fetcher is a
no-op. The username's presence is the opt-in.
* Connectivity gating — when the WebSocket primary is connected, the
fallback skips so it doesn't stomp fresher live data.
* Response parsing — successful, error, and empty AISHub payloads.
* Record normalization — bad records (no MMSI, sentinel positions) are
dropped without crashing.
* Merge behavior — records land in the shared ``_vessels`` dict with
``source: "aishub"`` and don't overwrite very-recent live updates.
* Poll interval clamping — env var overrides honored within [1, 360].
"""
from __future__ import annotations
import json
import os
import time
import pytest
# ---------------------------------------------------------------------------
# Configuration / gating
# ---------------------------------------------------------------------------
class TestGating:
def test_no_username_means_disabled(self, monkeypatch):
from services.fetchers.aishub_fallback import (
aishub_fallback_enabled,
fetch_aishub_vessels,
)
monkeypatch.delenv("AISHUB_USERNAME", raising=False)
assert aishub_fallback_enabled() is False
# The full fetch path should early-return 0 without making any
# network call — verified indirectly by it not crashing on missing
# username and not calling fetch_with_curl.
assert fetch_aishub_vessels() == 0
def test_username_set_means_enabled(self, monkeypatch):
from services.fetchers.aishub_fallback import aishub_fallback_enabled
monkeypatch.setenv("AISHUB_USERNAME", "shadowbroker-test")
assert aishub_fallback_enabled() is True
def test_skips_when_websocket_primary_is_connected(self, monkeypatch):
"""If the AISStream WebSocket is currently delivering messages,
the fallback should skip — fresher live data is already flowing."""
from services.fetchers import aishub_fallback
from services import ais_stream as ais
monkeypatch.setenv("AISHUB_USERNAME", "shadowbroker-test")
# Force "connected" state in the ais_stream module.
with ais._vessels_lock:
ais._last_msg_at = time.time() - 5 # 5s ago — well inside 60s
ais._proxy_spawn_count = 1
# Sanity check the gate:
assert ais.ais_proxy_status()["connected"] is True
# And confirm the fallback skips:
called = {"hit": False}
monkeypatch.setattr(
aishub_fallback,
"fetch_with_curl",
lambda *a, **kw: (_ for _ in ()).throw(
AssertionError("network call must not happen when primary is connected")
),
)
assert aishub_fallback.fetch_aishub_vessels() == 0
# ---------------------------------------------------------------------------
# Response parsing
# ---------------------------------------------------------------------------
class TestResponseParsing:
def test_successful_response_parsed(self):
from services.fetchers.aishub_fallback import _parse_aishub_response
payload = json.dumps([
{"ERROR": False, "USERNAME": "test", "FORMAT": "1", "RECORDS": 2},
[
{"MMSI": 123, "LATITUDE": 40.0, "LONGITUDE": -73.0},
{"MMSI": 456, "LATITUDE": 51.5, "LONGITUDE": -0.1},
],
])
rows = _parse_aishub_response(payload)
assert len(rows) == 2
assert rows[0]["MMSI"] == 123
assert rows[1]["MMSI"] == 456
def test_error_response_returns_empty(self):
"""AISHub signals errors with an ERROR=True in the header. We log
and treat as no data."""
from services.fetchers.aishub_fallback import _parse_aishub_response
payload = json.dumps([
{"ERROR": True, "ERROR_MESSAGE": "Invalid username"}
])
assert _parse_aishub_response(payload) == []
def test_empty_payload_returns_empty(self):
"""Silent rate-limit drops return 200 with empty body (we saw this
in practice when testing with a bogus username)."""
from services.fetchers.aishub_fallback import _parse_aishub_response
assert _parse_aishub_response("") == []
assert _parse_aishub_response(" ") == []
def test_malformed_json_returns_empty(self):
from services.fetchers.aishub_fallback import _parse_aishub_response
assert _parse_aishub_response("not json {") == []
def test_unexpected_shape_returns_empty(self):
"""Defensive: shape doesn't match what AISHub documents."""
from services.fetchers.aishub_fallback import _parse_aishub_response
assert _parse_aishub_response(json.dumps({"unexpected": "object"})) == []
assert _parse_aishub_response(json.dumps([])) == []
# Header-only with no records list:
assert _parse_aishub_response(json.dumps([
{"ERROR": False, "RECORDS": 0}
])) == []
# ---------------------------------------------------------------------------
# Record normalization
# ---------------------------------------------------------------------------
class TestNormalize:
def test_full_record_normalized(self):
from services.fetchers.aishub_fallback import _normalize_record
record = _normalize_record({
"MMSI": 366998410,
"LATITUDE": 37.8,
"LONGITUDE": -122.4,
"COG": 280,
"SOG": 12.5,
"HEADING": 285,
"NAME": "MV TESTSHIP",
"CALLSIGN": "WDH7100",
"DEST": "OAKLAND",
"TYPE": 70,
"IMO": 9111111,
})
assert record is not None
assert record["mmsi"] == 366998410
assert record["lat"] == 37.8
assert record["lng"] == -122.4
assert record["sog"] == 12.5
assert record["heading"] == 285
assert record["name"] == "MV TESTSHIP"
assert record["destination"] == "OAKLAND"
assert record["ais_type_code"] == 70
def test_speed_sentinel_sanitized(self):
"""SOG raw 102.3+ kn = "speed not available" in the AIS spec.
Sanitize to 0 so it doesn't look like a 200-knot ship."""
from services.fetchers.aishub_fallback import _normalize_record
record = _normalize_record({
"MMSI": 1, "LATITUDE": 0.5, "LONGITUDE": 0.5,
"SOG": 102.3, "COG": 0,
})
assert record["sog"] == 0.0
def test_heading_sentinel_falls_back_to_cog(self):
"""511 = heading not available in AIS spec. Use COG instead."""
from services.fetchers.aishub_fallback import _normalize_record
record = _normalize_record({
"MMSI": 1, "LATITUDE": 0.5, "LONGITUDE": 0.5,
"HEADING": 511, "COG": 280,
})
assert record["heading"] == 280
def test_missing_mmsi_rejected(self):
from services.fetchers.aishub_fallback import _normalize_record
assert _normalize_record({"LATITUDE": 0.5, "LONGITUDE": 0.5}) is None
assert _normalize_record({"MMSI": 0, "LATITUDE": 0.5, "LONGITUDE": 0.5}) is None
def test_no_position_rejected(self):
from services.fetchers.aishub_fallback import _normalize_record
assert _normalize_record({"MMSI": 1}) is None
assert _normalize_record({"MMSI": 1, "LATITUDE": 0.5}) is None
assert _normalize_record({"MMSI": 1, "LONGITUDE": 0.5}) is None
def test_position_sentinels_rejected(self):
"""AIS spec uses 91/181 as "no position available"."""
from services.fetchers.aishub_fallback import _normalize_record
assert _normalize_record({
"MMSI": 1, "LATITUDE": 91.0, "LONGITUDE": 0.0
}) is None
assert _normalize_record({
"MMSI": 1, "LATITUDE": 0.0, "LONGITUDE": 181.0
}) is None
def test_out_of_range_rejected(self):
from services.fetchers.aishub_fallback import _normalize_record
assert _normalize_record({
"MMSI": 1, "LATITUDE": 95.0, "LONGITUDE": 0.0
}) is None
assert _normalize_record({
"MMSI": 1, "LATITUDE": 0.0, "LONGITUDE": 200.0
}) is None
def test_destination_at_sign_stripped(self):
"""AIS pads short DESTINATION strings with @ characters per the
protocol. Strip them so the UI doesn't render "OAKLAND@@@@@"."""
from services.fetchers.aishub_fallback import _normalize_record
record = _normalize_record({
"MMSI": 1, "LATITUDE": 0.5, "LONGITUDE": 0.5,
"DEST": "OAKLAND@@@",
})
assert record["destination"] == "OAKLAND"
# ---------------------------------------------------------------------------
# Poll interval clamping
# ---------------------------------------------------------------------------
class TestPollInterval:
def test_default_is_twenty_minutes(self, monkeypatch):
from services.fetchers.aishub_fallback import aishub_poll_interval_minutes
monkeypatch.delenv("AISHUB_POLL_INTERVAL_MINUTES", raising=False)
assert aishub_poll_interval_minutes() == 20
def test_env_override_honored(self, monkeypatch):
from services.fetchers.aishub_fallback import aishub_poll_interval_minutes
monkeypatch.setenv("AISHUB_POLL_INTERVAL_MINUTES", "45")
assert aishub_poll_interval_minutes() == 45
def test_clamp_lower_bound(self, monkeypatch):
"""A 0 or negative env var would hammer the upstream — clamp."""
from services.fetchers.aishub_fallback import aishub_poll_interval_minutes
monkeypatch.setenv("AISHUB_POLL_INTERVAL_MINUTES", "0")
assert aishub_poll_interval_minutes() == 1
monkeypatch.setenv("AISHUB_POLL_INTERVAL_MINUTES", "-5")
assert aishub_poll_interval_minutes() == 1
def test_clamp_upper_bound(self, monkeypatch):
"""A 99999 env var would silence the fallback effectively forever."""
from services.fetchers.aishub_fallback import aishub_poll_interval_minutes
monkeypatch.setenv("AISHUB_POLL_INTERVAL_MINUTES", "99999")
assert aishub_poll_interval_minutes() == 360
def test_malformed_env_defaults(self, monkeypatch):
from services.fetchers.aishub_fallback import aishub_poll_interval_minutes
monkeypatch.setenv("AISHUB_POLL_INTERVAL_MINUTES", "twenty")
assert aishub_poll_interval_minutes() == 20
# ---------------------------------------------------------------------------
# End-to-end fetch + merge into _vessels store
# ---------------------------------------------------------------------------
class TestFetchAndMerge:
def _force_primary_disconnected(self):
"""Set ais_stream module state so the gate allows the fallback."""
from services import ais_stream as ais
with ais._vessels_lock:
# Far in the past → connected = false; spawn_count > 0 → primary
# has at least tried so the gate engages.
ais._last_msg_at = time.time() - 3600
ais._proxy_spawn_count = 5
ais._vessels.clear()
def test_vessels_merged_with_source_tag(self, monkeypatch):
"""Happy path: AISHub returns 2 ships, both land in ``_vessels``
with ``source: 'aishub'``."""
from services.fetchers import aishub_fallback
from services import ais_stream as ais
monkeypatch.setenv("AISHUB_USERNAME", "test-user")
self._force_primary_disconnected()
payload = json.dumps([
{"ERROR": False, "USERNAME": "test-user", "FORMAT": "1", "RECORDS": 2},
[
{
"MMSI": 111111111,
"LATITUDE": 40.0,
"LONGITUDE": -73.0,
"SOG": 12.0,
"COG": 270,
"HEADING": 275,
"NAME": "SHIP A",
"TYPE": 70,
},
{
"MMSI": 222222222,
"LATITUDE": 51.5,
"LONGITUDE": -0.1,
"SOG": 8.0,
"COG": 90,
"HEADING": 92,
"NAME": "SHIP B",
"TYPE": 60,
},
],
])
class FakeResp:
status_code = 200
text = payload
monkeypatch.setattr(
aishub_fallback, "fetch_with_curl", lambda *a, **kw: FakeResp()
)
count = aishub_fallback.fetch_aishub_vessels()
assert count == 2
with ais._vessels_lock:
v1 = ais._vessels.get(111111111)
v2 = ais._vessels.get(222222222)
assert v1 is not None
assert v1["source"] == "aishub"
assert v1["lat"] == 40.0
assert v1["name"] == "SHIP A"
assert v2 is not None
assert v2["source"] == "aishub"
assert v2["type"] == "passenger" # AIS type 60 → passenger
def test_does_not_overwrite_fresh_live_data(self, monkeypatch):
"""If the WebSocket pushed an update for an MMSI 0.5s ago and the
AISHub poll completes in that window, we should NOT clobber the
fresher live data."""
from services.fetchers import aishub_fallback
from services import ais_stream as ais
monkeypatch.setenv("AISHUB_USERNAME", "test-user")
self._force_primary_disconnected()
# Pre-seed _vessels with a "very fresh" live record.
fresh_ts = time.time()
with ais._vessels_lock:
ais._vessels[111111111] = {
"mmsi": 111111111,
"lat": 12.34,
"lng": 56.78,
"source": "aisstream",
"_updated": fresh_ts,
}
payload = json.dumps([
{"ERROR": False, "USERNAME": "test-user", "FORMAT": "1", "RECORDS": 1},
[
{
"MMSI": 111111111,
"LATITUDE": 99.0, # bogus to make the test obvious
"LONGITUDE": 99.0,
"NAME": "STALE",
"SOG": 0,
"COG": 0,
"TYPE": 0,
},
],
])
class FakeResp:
status_code = 200
text = payload
monkeypatch.setattr(
aishub_fallback, "fetch_with_curl", lambda *a, **kw: FakeResp()
)
# Note: 99.0/99.0 also exceeds the 91/181 sentinel guard and
# would be filtered. Pick a valid-but-bogus position instead.
payload = json.dumps([
{"ERROR": False, "USERNAME": "test-user", "FORMAT": "1", "RECORDS": 1},
[
{
"MMSI": 111111111,
"LATITUDE": 0.0, # different from the live 12.34
"LONGITUDE": 0.0,
"NAME": "STALE",
"SOG": 0,
"COG": 0,
"TYPE": 0,
},
],
])
monkeypatch.setattr(
aishub_fallback, "fetch_with_curl",
lambda *a, **kw: type("R", (), {"status_code": 200, "text": payload})(),
)
aishub_fallback.fetch_aishub_vessels()
with ais._vessels_lock:
v = ais._vessels.get(111111111)
# Live data wins — position should still be 12.34 / 56.78.
assert v["lat"] == 12.34
assert v["lng"] == 56.78
assert v["source"] == "aisstream"
def test_http_failure_returns_zero(self, monkeypatch):
from services.fetchers import aishub_fallback
monkeypatch.setenv("AISHUB_USERNAME", "test-user")
self._force_primary_disconnected()
class FailResp:
status_code = 503
text = ""
monkeypatch.setattr(
aishub_fallback, "fetch_with_curl", lambda *a, **kw: FailResp()
)
assert aishub_fallback.fetch_aishub_vessels() == 0
+258
View File
@@ -0,0 +1,258 @@
"""Cumulative fuel/CO2 tracking via per-aircraft observation timestamps.
Background
----------
Users want the running total of fuel burned per aircraft — not just the
rate. We track first-seen-at per icao24 and multiply elapsed observation
time by the model-based rate. This module's job is exclusively the
timestamp bookkeeping; multiplication happens in the flights/military
fetchers.
These tests pin:
* First sighting returns 0 (no airtime yet).
* Repeated sightings within ``REOPEN_GAP_S`` accumulate elapsed time.
* Gap longer than ``REOPEN_GAP_S`` resets the session (plane landed
and took off again — different flight).
* ``MAX_SESSION_SECONDS`` clamp protects against clock skew bugs.
* ``prune()`` drops stale entries.
* ``get_session_seconds`` reads without bumping last_seen.
* Empty / None icao input is a defensive no-op.
"""
from __future__ import annotations
import pytest
@pytest.fixture(autouse=True)
def _reset_observations():
from services.fetchers import flight_observations as obs
obs._reset_for_tests()
yield
obs._reset_for_tests()
class TestRecordObservation:
def test_first_sighting_returns_zero(self):
from services.fetchers.flight_observations import record_observation
assert record_observation("a12345", now=1000.0) == 0
def test_repeated_sightings_accumulate(self):
"""ADS-B refreshes every ~minute in practice, so each observation
is within ``REOPEN_GAP_S`` (15 min) of the last and we keep
accumulating. Walking the timestamps in 5-minute steps so we
stay inside the reopen window the whole way."""
from services.fetchers.flight_observations import record_observation
record_observation("a12345", now=1000.0)
# 1 minute later (within REOPEN_GAP_S)
assert record_observation("a12345", now=1060.0) == 60
# Step through 5-minute spaced refreshes — first_seen_at stays
# at 1000.0 the whole time, and we approach a 1-hour airtime.
assert record_observation("a12345", now=1360.0) == 360
assert record_observation("a12345", now=1660.0) == 660
assert record_observation("a12345", now=1960.0) == 960
assert record_observation("a12345", now=2260.0) == 1260
assert record_observation("a12345", now=2560.0) == 1560
assert record_observation("a12345", now=2860.0) == 1860
assert record_observation("a12345", now=3160.0) == 2160
assert record_observation("a12345", now=3460.0) == 2460
assert record_observation("a12345", now=3760.0) == 2760
assert record_observation("a12345", now=4060.0) == 3060
assert record_observation("a12345", now=4360.0) == 3360
# 1 hour after first sighting — still inside the 15-min reopen
# window from the prior 4360 observation.
assert record_observation("a12345", now=4600.0) == 3600
def test_gap_longer_than_reopen_resets_session(self):
"""If a hex hasn't been seen in ``REOPEN_GAP_S`` (15 min default),
the next sighting is treated as a new flight — first_seen_at resets."""
from services.fetchers.flight_observations import record_observation
record_observation("a12345", now=1000.0)
record_observation("a12345", now=1500.0) # 500s later — within gap
# Now 20 minutes of silence (1200s > 900s threshold) → session reset.
assert record_observation("a12345", now=2700.0) == 0
# And the next quick sighting starts accumulating from 2700 again.
assert record_observation("a12345", now=2760.0) == 60
def test_session_clamp(self):
"""Clock skew protection: when a hex has been continuously
observed for longer than ``MAX_SESSION_SECONDS``, clamp.
Synthesizes the state directly because driving 86,400+ seconds of
observations through the public API in a test would take 1000+
REOPEN_GAP_S-respecting steps.
"""
from services.fetchers import flight_observations as obs
from services.fetchers.flight_observations import _observations, _lock
# last_seen_at very recent so REOPEN_GAP_S branch does NOT fire,
# but first_seen_at way in the past so the elapsed math overflows
# MAX_SESSION_SECONDS. Clamp must kick in.
big_now = float(obs.MAX_SESSION_SECONDS + 1_000_000)
with _lock:
_observations["a12345"] = {
"first_seen_at": 0.0,
"last_seen_at": big_now - 60, # 60s ago — well inside gap window
}
elapsed = obs.record_observation("a12345", now=big_now)
assert elapsed == obs.MAX_SESSION_SECONDS, (
f"elapsed must be clamped to MAX_SESSION_SECONDS; got {elapsed}"
)
def test_empty_input_returns_zero(self):
from services.fetchers.flight_observations import record_observation
assert record_observation("") == 0
assert record_observation(None) == 0 # type: ignore[arg-type]
assert record_observation(" ") == 0
def test_case_insensitive_key(self):
"""ICAO24 hex codes are case-insensitive — adsb.lol lowercases
them, OpenSky may not. Normalize so both refer to the same airframe."""
from services.fetchers.flight_observations import record_observation
record_observation("A12345", now=1000.0)
# Different case must hit the same entry.
assert record_observation("a12345", now=1060.0) == 60
class TestGetSessionSeconds:
def test_read_only_does_not_bump(self):
from services.fetchers.flight_observations import (
record_observation,
get_session_seconds,
)
record_observation("a12345", now=1000.0)
record_observation("a12345", now=1060.0) # bumps last_seen
# Now read at t=2000. Without bumping, gap=2000-1060=940 > 900,
# so a recording call would reset. But the read should NOT reset.
seconds_at_2000 = get_session_seconds("a12345", now=2000.0)
assert seconds_at_2000 == 1000, (
f"read should return 2000-1000=1000s; got {seconds_at_2000}"
)
# Verify the next recording at t=2001 still resets (gap > 900s
# from the read above — proves the read didn't bump last_seen).
from services.fetchers.flight_observations import record_observation as rec
assert rec("a12345", now=2001.0) == 0 # session reset
def test_unknown_hex_returns_zero(self):
from services.fetchers.flight_observations import get_session_seconds
assert get_session_seconds("nonexistent") == 0
class TestPrune:
def test_drops_stale_entries(self):
from services.fetchers import flight_observations as obs
obs.record_observation("active", now=10_000.0)
obs.record_observation("stale", now=1.0)
dropped = obs.prune(now=10_000.0)
assert dropped == 1
# Active entry survives:
assert obs.get_session_seconds("active", now=10_001.0) == 1
# Stale entry was dropped — next obs starts fresh:
assert obs.record_observation("stale", now=10_002.0) == 0
def test_no_op_when_nothing_stale(self):
from services.fetchers import flight_observations as obs
obs.record_observation("hex1", now=1000.0)
obs.record_observation("hex2", now=1000.0)
dropped = obs.prune(now=1500.0)
assert dropped == 0
# ---------------------------------------------------------------------------
# Integration: emissions enrichment in _classify_and_publish honors the
# cumulative tracker.
# ---------------------------------------------------------------------------
class TestEmissionsCumulativeIntegration:
def _reset_store(self):
from services.fetchers._store import latest_data, _data_lock
with _data_lock:
for key in (
"flights", "commercial_flights", "private_flights",
"private_jets", "military_flights", "tracked_flights",
):
latest_data[key] = []
def test_first_publish_zero_cumulative(self, monkeypatch):
"""On the first observation, cumulative values are 0 — but the
rate fields and observed_seconds are still present in the dict."""
from services.fetchers import flights as flights_module
from services.fetchers._store import latest_data, _data_lock
self._reset_store()
monkeypatch.setattr(flights_module, "lookup_route", lambda _: None)
monkeypatch.setattr(flights_module, "lookup_aircraft_type", lambda _: "")
flights_module._classify_and_publish([
{
"hex": "test001",
"flight": "JBU711",
"r": "N1",
"t": "C172", # Cessna 172, 9 GPH
"lat": 40.0,
"lon": -100.0,
"alt_baro": 3000,
"gs": 100,
}
])
with _data_lock:
published = list(latest_data.get("flights", []))
assert len(published) == 1
emi = published[0].get("emissions")
assert emi is not None
assert emi["fuel_gph"] == 9
assert emi["observed_seconds"] == 0
assert emi["fuel_gallons_burned"] == 0.0
assert emi["co2_kg_emitted"] == 0.0
def test_second_publish_accumulates(self, monkeypatch):
"""Publishing the same hex a second time picks up real elapsed time
and produces non-zero cumulative values."""
import time as _time_real
from services.fetchers import flights as flights_module
from services.fetchers import flight_observations as obs
from services.fetchers._store import latest_data, _data_lock
self._reset_store()
monkeypatch.setattr(flights_module, "lookup_route", lambda _: None)
monkeypatch.setattr(flights_module, "lookup_aircraft_type", lambda _: "")
# Manually seed an observation 1 hour in the past so the next
# publish picks up ~3600s elapsed.
with obs._lock:
obs._observations["test002"] = {
"first_seen_at": _time_real.time() - 3600,
"last_seen_at": _time_real.time() - 60,
}
flights_module._classify_and_publish([
{
"hex": "test002",
"flight": "JBU711",
"r": "N1",
"t": "C172", # 9 GPH
"lat": 40.0,
"lon": -100.0,
"alt_baro": 3000,
"gs": 100,
}
])
with _data_lock:
published = list(latest_data.get("flights", []))
assert len(published) == 1
emi = published[0].get("emissions")
# Roughly 1 hour observed → 9 gal burned.
assert 3500 <= emi["observed_seconds"] <= 3700
assert 8.7 <= emi["fuel_gallons_burned"] <= 9.3
# CO2 = 9 gph * 9.57 kg/gal = 86.1 kg/hr.
assert 84 <= emi["co2_kg_emitted"] <= 88
+333
View File
@@ -0,0 +1,333 @@
"""GPS jamming detection — nac_p=0 counted, lowered thresholds.
Background
----------
Pre-fix, the detector had three stacked filters that together meant the
``gps_jamming`` layer almost never lit up:
1. ``nac_p == 0`` aircraft were dropped on the theory that "0 = old
transponder." But modern Mode-S Enhanced Surveillance transponders
also fall back to ``nac_p == 0`` when they lose GPS lock entirely —
which is *exactly* the jamming signature we want to catch.
2. ``GPS_JAMMING_MIN_AIRCRAFT = 5`` per 1°x1° cell.
3. ``GPS_JAMMING_MIN_RATIO = 0.30`` adjusted ratio.
Combined with the existing ``-1`` noise cushion (``adjusted = degraded - 1``)
the bar to clear required dense, busy airspace — but jamming hotspots
(eastern Med, eastern Ukraine, Iran/Iraq) tend to have sparser traffic
precisely because pilots avoid them.
These tests pin the new behavior:
* ``nac_p == 0`` is now counted as degraded.
* ``nac_p == None`` (no field — typical for OpenSky records) is still
skipped — absence isn't evidence.
* Thresholds lowered to 3 aircraft / 0.20 ratio.
* Public function signature accepts overrides so callers / future
operators can re-tune without code edits.
"""
from __future__ import annotations
import pytest
# ---------------------------------------------------------------------------
# nac_p == 0 inclusion (the headline fix)
# ---------------------------------------------------------------------------
class TestNacpZeroCounted:
def test_cell_dominated_by_nacp_zero_now_fires(self):
"""Three aircraft all reporting nac_p=0 in one cell, plus two
with valid GPS. Pre-fix the three nac_p=0 records were skipped
entirely (cell would have total=2, degraded=0, no zone). Post-fix
they count as degraded — this IS the jamming signature."""
from services.fetchers.flights import detect_gps_jamming_zones
# All in 1°x1° cell at int(lat)=40, int(lng)=-100
feed = [
{"hex": "a1", "lat": 40.1, "lng": -100.1, "nac_p": 0},
{"hex": "a2", "lat": 40.5, "lng": -100.5, "nac_p": 0},
{"hex": "a3", "lat": 40.9, "lng": -100.9, "nac_p": 0},
{"hex": "b1", "lat": 40.2, "lng": -100.3, "nac_p": 9},
{"hex": "b2", "lat": 40.7, "lng": -100.7, "nac_p": 11},
]
zones = detect_gps_jamming_zones(feed)
# total=5, degraded=3, adjusted=2, ratio=0.40 > 0.20 → zone fires.
assert len(zones) == 1
assert zones[0]["degraded"] == 3
assert zones[0]["total"] == 5
assert zones[0]["ratio"] == 0.40
# Grid-cell center coords.
assert zones[0]["lat"] == 40.5
assert zones[0]["lng"] == -99.5
def test_nacp_zero_alone_clears_min_aircraft(self):
"""A cell with exactly 3 aircraft all reporting nac_p=0 must
fire under the new MIN_AIRCRAFT=3 + MIN_RATIO=0.20 regime."""
from services.fetchers.flights import detect_gps_jamming_zones
feed = [
{"hex": "a1", "lat": 50.1, "lng": 30.1, "nac_p": 0},
{"hex": "a2", "lat": 50.5, "lng": 30.5, "nac_p": 0},
{"hex": "a3", "lat": 50.9, "lng": 30.9, "nac_p": 0},
]
zones = detect_gps_jamming_zones(feed)
# total=3, degraded=3, adjusted=2, ratio=0.667 > 0.20 → fires.
# severity is "medium" because 0.5 ≤ ratio < 0.75.
assert len(zones) == 1
assert zones[0]["severity"] == "medium"
# ---------------------------------------------------------------------------
# nac_p == None is still skipped (preserve OpenSky behavior)
# ---------------------------------------------------------------------------
class TestNoneStillSkipped:
def test_none_records_dont_add_to_grid(self):
"""OpenSky's /states/all doesn't include nac_p, so its records
arrive with the field absent (``rf.get("nac_p") is None``). These
records must NOT count toward total — absence-of-data isn't
evidence of either jamming OR working GPS."""
from services.fetchers.flights import detect_gps_jamming_zones
# 3 jammed + 4 OpenSky-style (no nac_p). Pre-fix and post-fix
# behavior should be identical here: None always skipped.
feed = [
{"hex": "a1", "lat": 40.1, "lng": -100.1, "nac_p": 0},
{"hex": "a2", "lat": 40.2, "lng": -100.2, "nac_p": 0},
{"hex": "a3", "lat": 40.3, "lng": -100.3, "nac_p": 0},
# OpenSky-style: no nac_p at all
{"hex": "o1", "lat": 40.4, "lng": -100.4},
{"hex": "o2", "lat": 40.5, "lng": -100.5},
{"hex": "o3", "lat": 40.6, "lng": -100.6},
{"hex": "o4", "lat": 40.7, "lng": -100.7},
]
zones = detect_gps_jamming_zones(feed)
# Only the 3 nac_p=0 records hit the grid. total=3, not 7.
assert len(zones) == 1
assert zones[0]["total"] == 3
assert zones[0]["degraded"] == 3
def test_explicit_none_skipped(self):
"""Same behavior when ``nac_p`` is present but set to None
(defensive — adsb.lol shouldn't do this, but downstream
normalizers might)."""
from services.fetchers.flights import detect_gps_jamming_zones
feed = [
{"hex": "a1", "lat": 0.1, "lng": 0.1, "nac_p": None},
{"hex": "a2", "lat": 0.2, "lng": 0.2, "nac_p": None},
{"hex": "a3", "lat": 0.3, "lng": 0.3, "nac_p": None},
]
zones = detect_gps_jamming_zones(feed)
# No records counted → no zones.
assert zones == []
# ---------------------------------------------------------------------------
# Lowered MIN_AIRCRAFT (5 → 3)
# ---------------------------------------------------------------------------
class TestMinAircraftLowered:
def test_three_aircraft_cell_now_qualifies(self):
"""Pre-fix MIN_AIRCRAFT=5 blocked sparse cells entirely. Post-fix
the bar is 3 aircraft per cell, which is realistic for the actual
jamming hotspots where traffic is thinner."""
from services.fetchers.flights import detect_gps_jamming_zones
feed = [
{"hex": "a1", "lat": 33.1, "lng": 44.1, "nac_p": 3},
{"hex": "a2", "lat": 33.2, "lng": 44.2, "nac_p": 5},
{"hex": "a3", "lat": 33.3, "lng": 44.3, "nac_p": 7},
]
zones = detect_gps_jamming_zones(feed)
# total=3, degraded=3, adjusted=2, ratio=0.667 — fires under new
# rules, would have been blocked by MIN_AIRCRAFT=5 pre-fix.
assert len(zones) == 1
def test_two_aircraft_cell_still_blocked(self):
"""We didn't lower the bar to 2 — that would create too much
single-transponder noise. Two aircraft per cell still doesn't
qualify."""
from services.fetchers.flights import detect_gps_jamming_zones
feed = [
{"hex": "a1", "lat": 33.1, "lng": 44.1, "nac_p": 3},
{"hex": "a2", "lat": 33.2, "lng": 44.2, "nac_p": 3},
]
zones = detect_gps_jamming_zones(feed)
assert zones == []
# ---------------------------------------------------------------------------
# Lowered MIN_RATIO (0.30 → 0.20)
# ---------------------------------------------------------------------------
class TestMinRatioLowered:
def test_ratio_between_old_and_new_threshold_fires(self):
"""Construct a cell whose ratio sits in the (0.20, 0.30) window:
fires under the new bar, would have been blocked pre-fix."""
from services.fetchers.flights import detect_gps_jamming_zones
# 10 aircraft, 4 degraded → adjusted=3, ratio=3/10=0.30.
# Pre-fix threshold was > 0.30 strict — would NOT fire.
# Post-fix threshold is > 0.20 — fires.
feed = (
[{"hex": f"d{i}", "lat": 40.1, "lng": -100.1, "nac_p": 3} for i in range(4)]
+ [{"hex": f"c{i}", "lat": 40.5, "lng": -100.5, "nac_p": 9} for i in range(6)]
)
zones = detect_gps_jamming_zones(feed)
assert len(zones) == 1
assert zones[0]["degraded"] == 4
assert zones[0]["total"] == 10
assert zones[0]["ratio"] == 0.30
def test_ratio_at_or_below_new_threshold_does_not_fire(self):
"""Ratio of exactly 0.20 must NOT fire (strict ``>`` comparison)."""
from services.fetchers.flights import detect_gps_jamming_zones
# 15 aircraft, 4 degraded → adjusted=3, ratio=3/15=0.20. Strictly
# not greater than 0.20, so doesn't qualify.
feed = (
[{"hex": f"d{i}", "lat": 40.1, "lng": -100.1, "nac_p": 3} for i in range(4)]
+ [{"hex": f"c{i}", "lat": 40.5, "lng": -100.5, "nac_p": 9} for i in range(11)]
)
zones = detect_gps_jamming_zones(feed)
assert zones == []
# ---------------------------------------------------------------------------
# Pre-existing noise cushion (-1) preserved
# ---------------------------------------------------------------------------
class TestNoiseCushionPreserved:
def test_single_quirky_transponder_doesnt_fire(self):
"""One degraded aircraft in a healthy cell shouldn't fire even
under the relaxed thresholds. The ``-1`` adjustment in the
detector exists for this reason."""
from services.fetchers.flights import detect_gps_jamming_zones
feed = (
[{"hex": "d1", "lat": 40.1, "lng": -100.1, "nac_p": 3}]
+ [{"hex": f"c{i}", "lat": 40.5, "lng": -100.5, "nac_p": 9} for i in range(10)]
)
zones = detect_gps_jamming_zones(feed)
# total=11, degraded=1, adjusted=0 → cell short-circuits.
assert zones == []
# ---------------------------------------------------------------------------
# Constants pinned (catches accidental rollback)
# ---------------------------------------------------------------------------
class TestConstantsPinned:
def test_min_aircraft_is_three(self):
from services.constants import GPS_JAMMING_MIN_AIRCRAFT
assert GPS_JAMMING_MIN_AIRCRAFT == 3, (
"MIN_AIRCRAFT must be 3; raising it back to 5 brings back the "
"'jamming never shows' bug."
)
def test_min_ratio_is_0_20(self):
from services.constants import GPS_JAMMING_MIN_RATIO
assert GPS_JAMMING_MIN_RATIO == 0.20, (
"MIN_RATIO must be 0.20; raising it back to 0.30 brings back "
"the 'jamming never shows' bug."
)
# ---------------------------------------------------------------------------
# Overrides honored
# ---------------------------------------------------------------------------
class TestOverridesHonored:
def test_overrides_supersede_constants(self):
"""The public signature accepts overrides so an operator can
re-tune at the call site (e.g. for a more aggressive setup in
an active conflict zone) without editing the module constants."""
from services.fetchers.flights import detect_gps_jamming_zones
feed = [
{"hex": "a1", "lat": 40.1, "lng": -100.1, "nac_p": 3},
{"hex": "a2", "lat": 40.2, "lng": -100.2, "nac_p": 3},
]
# With defaults (min_aircraft=3) this is blocked. With override=2 it fires.
assert detect_gps_jamming_zones(feed) == []
zones = detect_gps_jamming_zones(feed, min_aircraft=2)
assert len(zones) == 1
# ---------------------------------------------------------------------------
# lon vs lng compatibility
# ---------------------------------------------------------------------------
class TestLonLngCompat:
def test_lon_key_accepted(self):
"""adsb.lol records arrive with ``lon`` (no g). The OpenSky merge
normalizes to ``lng`` but raw records flowing into the detector
may use either. Make sure both work."""
from services.fetchers.flights import detect_gps_jamming_zones
feed = [
{"hex": "a1", "lat": 40.1, "lon": -100.1, "nac_p": 0},
{"hex": "a2", "lat": 40.2, "lon": -100.2, "nac_p": 0},
{"hex": "a3", "lat": 40.3, "lon": -100.3, "nac_p": 0},
]
zones = detect_gps_jamming_zones(feed)
assert len(zones) == 1
# ---------------------------------------------------------------------------
# Empty / malformed inputs don't crash
# ---------------------------------------------------------------------------
class TestRobustness:
def test_empty_feed(self):
from services.fetchers.flights import detect_gps_jamming_zones
assert detect_gps_jamming_zones([]) == []
def test_none_feed(self):
"""The wrapper at the call site passes ``raw_flights_snapshot``
which could in principle be None on a startup race. Handle it."""
from services.fetchers.flights import detect_gps_jamming_zones
assert detect_gps_jamming_zones(None) == []
def test_records_missing_position_skipped(self):
from services.fetchers.flights import detect_gps_jamming_zones
feed = [
{"hex": "noloc", "nac_p": 0},
{"hex": "nolat", "lng": -100.0, "nac_p": 0},
{"hex": "nolng", "lat": 40.0, "nac_p": 0},
]
assert detect_gps_jamming_zones(feed) == []
@@ -0,0 +1,252 @@
"""HF NUFORC fallback honors the rolling cutoff window.
Background
----------
The UAP sightings layer is sourced primarily from a live scrape of
nuforc.org. When that fails (Cloudflare 403, curl disabled on Windows,
wdtNonce regex stale, etc.) the code falls back to a static CSV mirror
hosted on Hugging Face at ``kcimc/NUFORC/nuforc_str.csv``.
The HF mirror is maintained by a third party and refreshed sporadically.
Pre-fix, the fallback parsed every row, sorted by ``occurred`` descending,
and took the top 250 — **with no date cutoff**. When the HF mirror is
stale (its "newest" rows are ~2-3 years old), users saw a map full of
2022-2023 sightings labeled as the "last 60 days" layer.
These tests pin the new behavior:
* Rows older than ``_NUFORC_RECENT_DAYS`` are dropped before the take-top-N.
* If the HF mirror has nothing in the window, the fallback returns ``[]``
and logs ERROR (don't silently serve stale data).
* ``fetch_uap_sightings`` records the failure when BOTH paths fail, so
the layer shows as broken in the health registry instead of "fresh".
"""
from __future__ import annotations
import logging
from datetime import datetime as real_datetime
class _FixedDateTime(real_datetime):
"""A datetime whose utcnow() returns a pinned value, for deterministic
cutoff math. Subclasses real datetime so existing operations still work."""
@classmethod
def utcnow(cls):
return cls(2026, 5, 1, 12, 0, 0)
class _StubResponse:
status_code = 200
def __init__(self, text: str):
self.text = text
def _stub_geocode_cache(*_args, **_kwargs):
"""Pre-populated location cache so the fallback doesn't try to hit
Photon during the test."""
return {
"Denver, CO, USA": [39.7392, -104.9903],
"Seattle, WA, USA": [47.6062, -122.3321],
"Phoenix, AZ, USA": [33.4484, -112.0740],
}
def test_hf_fallback_drops_rows_older_than_60_days(monkeypatch):
"""Pre-fix: a row from 2023 would make it into the layer if it was
among the newest 250 in the HF mirror. Post-fix: it's filtered out
before we even count to 250."""
from services.fetchers import earth_observation as eo
# 2026-05-01 - 60 days = 2026-03-02. So 2026-03-01 is one day too old.
csv_text = (
"Sighting,Occurred,Location,Shape,Duration,Posted,Summary\n"
'1,2026-04-15 21:00:00 Local,"Denver, CO, USA",Triangle,5 minutes,2026-04-16,"In-window sighting"\n'
'2,2023-06-01 21:00:00 Local,"Seattle, WA, USA",Light,30 seconds,2023-06-02,"Three years old"\n'
'3,2022-01-15 20:00:00 Local,"Phoenix, AZ, USA",Disk,2 minutes,2022-01-16,"Even older"\n'
)
monkeypatch.setattr(eo, "datetime", _FixedDateTime)
monkeypatch.setattr(eo, "fetch_with_curl", lambda *a, **kw: _StubResponse(csv_text))
monkeypatch.setattr(eo, "_load_nuforc_location_cache", _stub_geocode_cache)
monkeypatch.setattr(eo, "_save_nuforc_location_cache", lambda cache: None)
# If the cutoff is missing, the geocoder may still get called for the
# 2022/2023 rows. We assert geocoder is NEVER invoked for stale rows.
geocode_calls: list[str] = []
def _geocode_spy(location, city, state, country=""):
geocode_calls.append(location)
return None # already in cache, shouldn't be hit anyway
monkeypatch.setattr(eo, "_geocode_uap_location", _geocode_spy)
sightings = eo._build_uap_sightings_from_hf_mirror()
ids = [s["id"] for s in sightings]
assert ids == ["NUFORC-1"], f"only the 2026 row should survive: got {ids}"
# Stale rows must not have been geocoded — they should be dropped
# before the geocoding loop is reached.
assert geocode_calls == []
def test_hf_fallback_returns_empty_when_mirror_is_fully_stale(monkeypatch, caplog):
"""The smoking-gun case: the HF mirror is so stale that NO rows are
within the rolling window. Pre-fix returned 250 ancient rows. Post-fix
returns ``[]`` and logs ERROR so the operator knows the layer is dead."""
from services.fetchers import earth_observation as eo
csv_text = (
"Sighting,Occurred,Location,Shape,Duration,Posted,Summary\n"
'1,2023-04-15 21:00:00 Local,"Denver, CO, USA",Triangle,5 minutes,2023-04-16,"Old"\n'
'2,2022-06-01 21:00:00 Local,"Seattle, WA, USA",Light,30 seconds,2022-06-02,"Older"\n'
'3,2021-01-15 20:00:00 Local,"Phoenix, AZ, USA",Disk,2 minutes,2021-01-16,"Ancient"\n'
)
monkeypatch.setattr(eo, "datetime", _FixedDateTime)
monkeypatch.setattr(eo, "fetch_with_curl", lambda *a, **kw: _StubResponse(csv_text))
monkeypatch.setattr(eo, "_load_nuforc_location_cache", _stub_geocode_cache)
monkeypatch.setattr(eo, "_save_nuforc_location_cache", lambda cache: None)
monkeypatch.setattr(eo, "_geocode_uap_location", lambda *a, **kw: None)
with caplog.at_level(logging.ERROR, logger="services.fetchers.earth_observation"):
sightings = eo._build_uap_sightings_from_hf_mirror()
assert sightings == []
# The error log should mention how many stale rows were dropped so the
# operator can tell the mirror is the problem (not "we got 0 rows" which
# could also mean the download failed).
relevant = [r for r in caplog.records if "HF fallback yielded 0 rows" in r.getMessage()]
assert relevant, "expected loud ERROR when HF mirror is fully stale"
# The message should report the count of dropped stale rows.
assert any("dropped 3" in r.getMessage() for r in relevant)
def test_hf_fallback_still_returns_data_when_some_rows_are_in_window(monkeypatch):
"""Mixed-age mirror: some rows in the window, some not. The fallback
should return only the in-window rows and not log the doomsday ERROR."""
from services.fetchers import earth_observation as eo
csv_text = (
"Sighting,Occurred,Location,Shape,Duration,Posted,Summary\n"
'1,2026-04-15 21:00:00 Local,"Denver, CO, USA",Triangle,5 minutes,2026-04-16,"Fresh"\n'
'2,2026-04-10 21:00:00 Local,"Seattle, WA, USA",Light,30 seconds,2026-04-10,"Also fresh"\n'
'3,2020-01-15 20:00:00 Local,"Phoenix, AZ, USA",Disk,2 minutes,2020-01-16,"Ancient"\n'
)
monkeypatch.setattr(eo, "datetime", _FixedDateTime)
monkeypatch.setattr(eo, "fetch_with_curl", lambda *a, **kw: _StubResponse(csv_text))
monkeypatch.setattr(eo, "_load_nuforc_location_cache", _stub_geocode_cache)
monkeypatch.setattr(eo, "_save_nuforc_location_cache", lambda cache: None)
monkeypatch.setattr(eo, "_geocode_uap_location", lambda *a, **kw: None)
sightings = eo._build_uap_sightings_from_hf_mirror()
ids = sorted(s["id"] for s in sightings)
assert ids == ["NUFORC-1", "NUFORC-2"], f"only in-window rows should appear: got {ids}"
def test_fetch_uap_sightings_marks_failure_when_both_paths_empty(monkeypatch, caplog):
"""When the live path raises AND the HF fallback returns empty,
``fetch_uap_sightings`` must:
* NOT mark the layer fresh (pre-fix bug: it did, so the layer
showed as healthy-but-empty for days)
* call ``assert_canary("uap_sightings", 0)`` so the health
registry surfaces the broken layer
* log an ERROR with the live-path exception for debugging
"""
from services.fetchers import earth_observation as eo
from services.fetchers import _store
monkeypatch.setattr(_store, "is_any_active", lambda layer: True)
monkeypatch.setattr(eo, "_load_nuforc_sightings_cache", lambda force_refresh=False: None)
def _boom():
raise RuntimeError("NUFORC live: zero rows pulled across 3 months")
monkeypatch.setattr(eo, "_build_recent_uap_sightings", _boom)
monkeypatch.setattr(eo, "_build_uap_sightings_from_hf_mirror", lambda: [])
marked: list[str] = []
monkeypatch.setattr(eo, "_mark_fresh", lambda *keys: marked.extend(keys))
canary_calls: list[tuple[str, int]] = []
import services.slo as slo
monkeypatch.setattr(
slo, "assert_canary", lambda key, value: canary_calls.append((key, int(value)))
)
with caplog.at_level(logging.ERROR, logger="services.fetchers.earth_observation"):
eo.fetch_uap_sightings()
assert marked == [], "broken layer must NOT be marked fresh"
assert canary_calls == [("uap_sightings", 0)], (
f"expected canary trip when both paths fail; got {canary_calls}"
)
# The live error message should propagate into the error log so the
# operator can tell live failed AND fallback was empty (not the other
# way around).
assert any(
"both live NUFORC and HF fallback" in r.getMessage()
for r in caplog.records
)
def test_fetch_uap_sightings_succeeds_when_fallback_returns_data(monkeypatch):
"""Positive path: live fails, fallback returns rows. The layer is
populated and marked fresh; assert_canary is NOT tripped (we only
trip the canary when the layer has zero data)."""
from services.fetchers import earth_observation as eo
from services.fetchers import _store
monkeypatch.setattr(_store, "is_any_active", lambda layer: True)
monkeypatch.setattr(eo, "_load_nuforc_sightings_cache", lambda force_refresh=False: None)
monkeypatch.setattr(
eo, "_build_recent_uap_sightings", lambda: (_ for _ in ()).throw(RuntimeError("live down"))
)
fallback_rows = [{"id": "NUFORC-fb-1", "date_time": "2026-04-20", "lat": 0.0, "lng": 0.0}]
monkeypatch.setattr(eo, "_build_uap_sightings_from_hf_mirror", lambda: fallback_rows)
monkeypatch.setattr(eo, "_save_nuforc_sightings_cache", lambda s: None)
marked: list[str] = []
monkeypatch.setattr(eo, "_mark_fresh", lambda *keys: marked.extend(keys))
canary_calls: list[tuple[str, int]] = []
import services.slo as slo
monkeypatch.setattr(
slo, "assert_canary", lambda key, value: canary_calls.append((key, int(value)))
)
eo.fetch_uap_sightings()
assert marked == ["uap_sightings"]
assert canary_calls == [], "canary should not trip when fallback supplies data"
def test_uap_scheduler_runs_weekly_not_daily():
"""The cron job for the UAP layer must be configured for Mondays at
12:00 UTC, not daily. Daily was the pre-fix default; weekly matches
the layer's stated cadence (a rolling 60-day digest) and keeps load
on nuforc.org light."""
from services import data_fetcher
src = data_fetcher.__file__
with open(src, "r", encoding="utf-8") as f:
text = f.read()
# Anchor on the scheduler block by id, then assert the cron triggers.
assert "uap_sightings_weekly" in text, (
"scheduler id should be uap_sightings_weekly (was uap_sightings_daily pre-fix)"
)
# The day_of_week directive is the difference between daily and weekly.
# If somebody flips it back to daily, this fires.
weekly_block = text.split("uap_sightings_weekly", 1)[0]
# Walk backwards for the matching add_job call.
add_job_idx = weekly_block.rfind("add_job(")
assert add_job_idx >= 0, "could not locate add_job block for UAP scheduler"
job_block = text[add_job_idx : text.find(")", text.index("uap_sightings_weekly")) + 1]
assert 'day_of_week="mon"' in job_block, (
f"expected day_of_week='mon' in UAP scheduler block:\n{job_block}"
)
+6
View File
@@ -39,6 +39,7 @@ import { useFeedHealth } from '@/hooks/useFeedHealth';
import { useKeyboardShortcuts } from '@/hooks/useKeyboardShortcuts';
import KeyboardShortcutsOverlay from '@/components/KeyboardShortcutsOverlay';
import AlertToast from '@/components/AlertToast';
import AisUpstreamBanner from '@/components/AisUpstreamBanner';
import { useAlertToasts } from '@/hooks/useAlertToasts';
import { useWatchlist } from '@/hooks/useWatchlist';
import WatchlistWidget from '@/components/WatchlistWidget';
@@ -933,6 +934,11 @@ export default function Dashboard() {
onFlyTo={handleFlyTo}
/>
{/* AIS UPSTREAM OUTAGE BANNER — renders only when AIS is configured
but the WebSocket upstream is unreachable. Tells users the empty
ocean isn't their fault. */}
<AisUpstreamBanner />
{/* ONBOARDING MODAL */}
{showOnboarding && (
<OnboardingModal
@@ -0,0 +1,61 @@
/**
* AisUpstreamBanner — visible notice that AIS ship data is unavailable
* because the upstream provider (AISStream) is offline.
*
* Renders nothing when AIS is healthy or when AIS isn't configured at all.
* Mounted at the app shell level so users see it before they wonder why
* the ocean looks empty.
*/
import { useState } from 'react';
import { useAisUpstreamHealth } from '@/hooks/useAisUpstreamHealth';
export function AisUpstreamBanner() {
const health = useAisUpstreamHealth();
const [dismissed, setDismissed] = useState(false);
if (!health || !health.aisEnabled || health.connected || dismissed) {
return null;
}
// Format the staleness for the operator. ``null`` means we never received
// anything since startup; otherwise show minutes if > 60s.
let stalenessLabel = 'never received';
if (health.lastMsgAgeSeconds != null) {
const minutes = Math.floor(health.lastMsgAgeSeconds / 60);
if (minutes >= 1) {
stalenessLabel = `last update ${minutes} min ago`;
} else {
stalenessLabel = `last update ${health.lastMsgAgeSeconds}s ago`;
}
}
return (
<div
role="status"
aria-live="polite"
className="pointer-events-auto fixed top-3 left-1/2 z-[100] -translate-x-1/2 max-w-[640px] rounded-md border border-amber-500/60 bg-amber-900/85 px-4 py-2 text-sm text-amber-50 shadow-lg backdrop-blur"
>
<div className="flex items-start gap-3">
<span aria-hidden className="mt-0.5 text-amber-300"></span>
<div className="flex-1">
<div className="font-semibold">Ship data temporarily unavailable</div>
<div className="text-xs opacity-90">
AISStream upstream is offline ({stalenessLabel}). The map will
refill once their service comes back online nothing is wrong
with your install.
</div>
</div>
<button
type="button"
onClick={() => setDismissed(true)}
aria-label="Dismiss"
className="text-amber-200 hover:text-white"
>
</button>
</div>
</div>
);
}
export default AisUpstreamBanner;
+47 -11
View File
@@ -249,34 +249,70 @@ const VESSEL_TYPE_WIKI: Record<string, string> = {
type FlightTrailPoint = { lat?: number; lng?: number; alt?: number; ts?: number } | number[];
function formatObservedDuration(seconds: number): string {
// Compact "1h 14m" / "23m" / "45s" — matches the density of the rest
// of the flight tooltip. < 60s is shown as "<1m" so the user knows
// we've JUST started observing this hex (cumulative will still be 0).
if (!Number.isFinite(seconds) || seconds <= 0) return '<1m';
if (seconds < 60) return '<1m';
const totalMinutes = Math.floor(seconds / 60);
const hours = Math.floor(totalMinutes / 60);
const minutes = totalMinutes % 60;
if (hours > 0) return `${hours}h ${minutes}m`;
return `${minutes}m`;
}
function EmissionsEstimateBlock({ flight }: { flight: any }) {
const emissions = flight?.emissions;
const context = emissions ? 'Model-based cruise estimate' : null;
// Cumulative fuel/CO2 since the backend first saw this hex this
// flight session. Prefer these big numbers — the user explicitly
// wanted "the actual fuel that has been burned", not the rate.
// Rates are still shown below as smaller context.
const observedSec = Number(emissions?.observed_seconds ?? 0);
const fuelBurned = Number(emissions?.fuel_gallons_burned ?? 0);
const co2Emitted = Number(emissions?.co2_kg_emitted ?? 0);
const haveCumulative = emissions && observedSec > 0;
return (
<div className="border-b border-[var(--border-primary)] pb-2">
<span className="text-[var(--text-muted)] text-[10px] block mb-1.5">EMISSIONS ESTIMATE</span>
<div className="flex gap-3">
<div className="flex-1 bg-[var(--bg-primary)]/50 border border-[var(--border-primary)] px-2 py-1.5">
<div className="text-[11px] text-[var(--text-muted)] tracking-widest">FUEL RATE</div>
<div className="text-xs font-bold text-orange-400">
{emissions ? (
<>{emissions.fuel_gph} <span className="text-[11px] text-[var(--text-muted)] font-normal">GPH</span></>
<div className="text-[11px] text-[var(--text-muted)] tracking-widest">FUEL BURNED</div>
<div className="text-sm font-bold text-orange-400">
{haveCumulative ? (
<>{fuelBurned.toLocaleString(undefined, { maximumFractionDigits: 1 })} <span className="text-[11px] text-[var(--text-muted)] font-normal">gal</span></>
) : emissions ? (
<span className="text-[var(--text-muted)] font-normal text-xs"></span>
) : 'UNKNOWN'}
</div>
{emissions && (
<div className="text-[10px] text-[var(--text-muted)] mt-0.5">
@ {emissions.fuel_gph} gph
</div>
)}
</div>
<div className="flex-1 bg-[var(--bg-primary)]/50 border border-[var(--border-primary)] px-2 py-1.5">
<div className="text-[11px] text-[var(--text-muted)] tracking-widest">CO2 RATE</div>
<div className="text-xs font-bold text-red-400">
{emissions ? (
<>{emissions.co2_kg_per_hour.toLocaleString()} <span className="text-[11px] text-[var(--text-muted)] font-normal">KG/HR</span></>
<div className="text-[11px] text-[var(--text-muted)] tracking-widest">CO2 EMITTED</div>
<div className="text-sm font-bold text-red-400">
{haveCumulative ? (
<>{co2Emitted.toLocaleString(undefined, { maximumFractionDigits: 1 })} <span className="text-[11px] text-[var(--text-muted)] font-normal">kg</span></>
) : emissions ? (
<span className="text-[var(--text-muted)] font-normal text-xs"></span>
) : 'UNKNOWN'}
</div>
{emissions && (
<div className="text-[10px] text-[var(--text-muted)] mt-0.5">
@ {emissions.co2_kg_per_hour.toLocaleString()} kg/hr
</div>
)}
</div>
</div>
{context && (
{emissions && (
<div className="mt-1.5 text-[10px] text-[var(--text-muted)] leading-relaxed">
{context}
{haveCumulative
? `Observed in flight for ${formatObservedDuration(observedSec)} · model-based cruise estimate`
: 'Just observed · totals will appear on next refresh'}
</div>
)}
</div>
@@ -0,0 +1,85 @@
/**
* useAisUpstreamHealth — polls /api/health and exposes AIS proxy connectivity.
*
* Background: AISStream's WebSocket server went fully offline 2026-05-23 (TCP
* timeouts at stream.aisstream.io). The backend kept reconnecting in a tight
* loop and the ships layer silently went empty. Users had no signal that the
* problem was upstream, not their config. This hook surfaces the state so a
* banner can explain "AIS upstream is offline" instead of letting users
* wonder.
*
* The poll interval is intentionally relaxed (30s) — this is a low-urgency UX
* signal, not a real-time data feed. Backend already escalates top_status to
* "degraded" when AIS is configured-but-disconnected.
*/
import { useEffect, useRef, useState } from 'react';
import { API_BASE } from '@/lib/api';
export interface AisUpstreamHealth {
/** True when we've received a vessel message in the last ~60s. */
connected: boolean;
/** Seconds since the last vessel message; null when we've never seen one. */
lastMsgAgeSeconds: number | null;
/**
* True when the SPKI-pinned fallback is in effect (issue #258).
* Data still flows in this mode — it's a separate, less urgent signal
* than ``connected``.
*/
degradedTls: boolean;
/** How many times the proxy has been spawned (sustained growth without
* ``connected`` means upstream is dead and we're respawning in a loop). */
proxySpawnCount: number;
/** Whether the operator has configured an API key. When false, the banner
* shouldn't fire because "AIS is off" is the intended state. The backend
* signals this via the ``connected`` flag being false AND no msg ever
* seen — we approximate it by requiring at least one spawn before
* declaring an outage. */
aisEnabled: boolean;
}
const POLL_INTERVAL_MS = 30_000;
export function useAisUpstreamHealth(): AisUpstreamHealth | null {
const [health, setHealth] = useState<AisUpstreamHealth | null>(null);
const cancelledRef = useRef(false);
useEffect(() => {
cancelledRef.current = false;
const fetchHealth = async () => {
try {
const res = await fetch(`${API_BASE}/api/health`, { cache: 'no-store' });
if (!res.ok) return;
const body = await res.json();
if (cancelledRef.current) return;
const proxy = body?.ais_proxy ?? {};
// ``proxy_spawn_count > 0`` is the cheapest "AIS is enabled" check:
// if the backend never spawned the proxy (no API key, opt-out env)
// we shouldn't ever show the outage banner. Once the proxy has
// spawned at least once we know the operator wants AIS data.
const spawns = Number(proxy.proxy_spawn_count ?? 0);
setHealth({
connected: Boolean(proxy.connected),
lastMsgAgeSeconds:
proxy.last_msg_age_seconds == null
? null
: Number(proxy.last_msg_age_seconds),
degradedTls: Boolean(proxy.degraded_tls),
proxySpawnCount: spawns,
aisEnabled: spawns > 0,
});
} catch {
// Backend unreachable — separate problem. Banner not relevant.
}
};
void fetchHealth();
const interval = setInterval(() => void fetchHealth(), POLL_INTERVAL_MS);
return () => {
cancelledRef.current = true;
clearInterval(interval);
};
}, []);
return health;
}