Fix #288: viewport-scoped live-data for heavy layers only (#294)

Reported by @tg12 in the external security/correctness audit.

Before this change, /api/live-data/{fast,slow} accepted s/w/n/e query
params but their Query() descriptions explicitly said "(ignored)". The
endpoints shipped the full in-memory world dataset on every poll:

    /api/live-data/fast → 16.88 MB
    /api/live-data/slow → 10.12 MB
                          ── 27 MB per poll cycle, regardless of zoom

For a node with N operators each polling at the steady 15s/120s cadence,
this is hundreds of MB/minute of outbound traffic that never gets used —
the GPU just culls everything outside the viewport client-side. On a
Tor-bridged or LTE-backed node, that bandwidth bill is the actual cost.

This change makes the existing s/w/n/e params honored — when all four
bounds are supplied, the backend bbox-filters a curated set of heavy,
density-driven, time-sensitive collections to that viewport (with the
existing 20% padding from _bbox_filter):

    /fast: commercial_flights, military_flights, private_flights,
           private_jets, tracked_flights, ships, cctv, uavs, liveuamap,
           gps_jamming, sigint, trains
    /slow: gdelt, firms_fires, kiwisdr, scanners, psk_reporter

Static reference layers (satellites, datacenters, military_bases,
power_plants, satnogs, weather, news, stocks, etc.) deliberately STAY
world-scale so panning never reveals an "empty world" of infrastructure.
That preserves the no-hostile-UX feel of the existing dashboard.

Behavior contract:

  * Without bbox params (or with a partial bbox), the response is
    byte-for-byte identical to the pre-#288 implementation. No
    behavior change for any existing caller that hasn't opted in.
  * World-scale bbox (lng_span >= 300 or lat_span >= 120) short-circuits
    filtering and shares the global ETag — zoomed-out operators all
    hit the same 304 cache exactly like before.
  * ETag now mixes a 1°-quantized bbox suffix when filtering engages,
    so two viewports never poison each other's 304 cache. Sub-degree
    pans land in the same ETag bucket (i.e. don't bust the cache on
    every mouse drag).

Polling cadence, rate-limit windows, and the 304 short-circuit are all
unchanged. Only the SIZE of the responses changes, and only when the
caller opts in via bounds.

Frontend wiring: useViewportBounds reuses the same coarsened/
expanded bounds it already computes for the AIS /api/viewport POST and
pushes them into a new module-level liveDataViewport store.
useDataPolling reads from that store via appendLiveDataBoundsParams
when building each live-data URL.

Tests cover: no-bbox → world data; bbox → heavy layers filtered;
bbox → reference layers untouched; world-scale bbox → no filter;
partial bbox → treated as no bbox; ETag changes with bbox; sub-degree
pan → same ETag; 304 path works; antimeridian-crossing bbox handled.

Co-authored-by: BigBodyCobain <moatbc@gmail.com>
This commit is contained in:
Shadowbroker
2026-05-22 00:56:29 -06:00
committed by GitHub
co-authored by BigBodyCobain
parent 35cd4e4c71
commit 19fb7f0b1e
5 changed files with 494 additions and 20 deletions
+20 -10
View File
@@ -1,6 +1,7 @@
import { useEffect, useRef } from "react";
import { API_BASE } from "@/lib/api";
import { mergeData, setBackendStatus as setStoreBackendStatus } from "./useDataStore";
import { appendLiveDataBoundsParams } from "@/lib/liveDataViewport";
export type BackendStatus = 'connecting' | 'connected' | 'disconnected';
@@ -32,8 +33,8 @@ export async function forceRefreshLiveData(): Promise<void> {
try {
const [fastRes, slowRes] = await Promise.all([
fetch(`${API_BASE}/api/live-data/fast`),
fetch(`${API_BASE}/api/live-data/slow`),
fetch(appendLiveDataBoundsParams(`${API_BASE}/api/live-data/fast`)),
fetch(appendLiveDataBoundsParams(`${API_BASE}/api/live-data/slow`)),
]);
if (fastRes.ok) {
@@ -85,9 +86,13 @@ export const LAYER_TOGGLE_EVENT = 'sb:layer-toggle';
/**
* Polls the backend for fast and slow data tiers.
*
* All data is fetched globally (no bbox filtering) — the backend returns its
* full in-memory cache and MapLibre culls off-screen entities on the GPU.
* This eliminates the "empty map when zooming out" lag.
* Issue #288: heavy, density-driven layers (vessels, aircraft, gdelt
* events, fires, sigint, …) are bbox-scoped to the visible map area via
* `appendLiveDataBoundsParams`. Static reference layers (datacenters,
* military bases, power plants, satellites, weather, news, …) are NOT
* filtered backend-side, so panning never reveals an "empty world" of
* infrastructure. World-zoomed views skip bbox params entirely and hit
* the shared ETag cache exactly like the pre-#288 behaviour.
*
* The AIS stream viewport POST (/api/viewport) is still handled separately
* by useViewportBounds to limit upstream AIS ingestion.
@@ -147,7 +152,9 @@ export function useDataPolling() {
const useStartupPayload = !fetchedStartupFastPayload && !fastEtag.current;
const headers: Record<string, string> = {};
if (!useStartupPayload && fastEtag.current) headers['If-None-Match'] = fastEtag.current;
const url = `${API_BASE}/api/live-data/fast${useStartupPayload ? '?initial=1' : ''}`;
const url = appendLiveDataBoundsParams(
`${API_BASE}/api/live-data/fast${useStartupPayload ? '?initial=1' : ''}`,
);
const res = await fetch(url, {
headers,
signal: controller.signal,
@@ -193,10 +200,13 @@ export function useDataPolling() {
try {
const headers: Record<string, string> = {};
if (slowEtag.current) headers['If-None-Match'] = slowEtag.current;
const res = await fetch(`${API_BASE}/api/live-data/slow`, {
headers,
signal: controller.signal,
});
const res = await fetch(
appendLiveDataBoundsParams(`${API_BASE}/api/live-data/slow`),
{
headers,
signal: controller.signal,
},
);
if (res.status === 304) { scheduleNext('slow'); return; }
if (res.ok) {
slowEtag.current = res.headers.get('etag') || null;