mirror of
https://github.com/BigBodyCobain/Shadowbroker.git
synced 2026-09-27 11:11:50 +02:00
Reported by @tg12 in the external security/correctness audit.
Before this change, /api/live-data/{fast,slow} accepted s/w/n/e query
params but their Query() descriptions explicitly said "(ignored)". The
endpoints shipped the full in-memory world dataset on every poll:
/api/live-data/fast → 16.88 MB
/api/live-data/slow → 10.12 MB
── 27 MB per poll cycle, regardless of zoom
For a node with N operators each polling at the steady 15s/120s cadence,
this is hundreds of MB/minute of outbound traffic that never gets used —
the GPU just culls everything outside the viewport client-side. On a
Tor-bridged or LTE-backed node, that bandwidth bill is the actual cost.
This change makes the existing s/w/n/e params honored — when all four
bounds are supplied, the backend bbox-filters a curated set of heavy,
density-driven, time-sensitive collections to that viewport (with the
existing 20% padding from _bbox_filter):
/fast: commercial_flights, military_flights, private_flights,
private_jets, tracked_flights, ships, cctv, uavs, liveuamap,
gps_jamming, sigint, trains
/slow: gdelt, firms_fires, kiwisdr, scanners, psk_reporter
Static reference layers (satellites, datacenters, military_bases,
power_plants, satnogs, weather, news, stocks, etc.) deliberately STAY
world-scale so panning never reveals an "empty world" of infrastructure.
That preserves the no-hostile-UX feel of the existing dashboard.
Behavior contract:
* Without bbox params (or with a partial bbox), the response is
byte-for-byte identical to the pre-#288 implementation. No
behavior change for any existing caller that hasn't opted in.
* World-scale bbox (lng_span >= 300 or lat_span >= 120) short-circuits
filtering and shares the global ETag — zoomed-out operators all
hit the same 304 cache exactly like before.
* ETag now mixes a 1°-quantized bbox suffix when filtering engages,
so two viewports never poison each other's 304 cache. Sub-degree
pans land in the same ETag bucket (i.e. don't bust the cache on
every mouse drag).
Polling cadence, rate-limit windows, and the 304 short-circuit are all
unchanged. Only the SIZE of the responses changes, and only when the
caller opts in via bounds.
Frontend wiring: useViewportBounds reuses the same coarsened/
expanded bounds it already computes for the AIS /api/viewport POST and
pushes them into a new module-level liveDataViewport store.
useDataPolling reads from that store via appendLiveDataBoundsParams
when building each live-data URL.
Tests cover: no-bbox → world data; bbox → heavy layers filtered;
bbox → reference layers untouched; world-scale bbox → no filter;
partial bbox → treated as no bbox; ETag changes with bbox; sub-degree
pan → same ETag; 304 path works; antimeridian-crossing bbox handled.
Co-authored-by: BigBodyCobain <moatbc@gmail.com>
This commit is contained in:
co-authored by
BigBodyCobain
parent
35cd4e4c71
commit
19fb7f0b1e
@@ -1,6 +1,7 @@
|
||||
import { useEffect, useRef } from "react";
|
||||
import { API_BASE } from "@/lib/api";
|
||||
import { mergeData, setBackendStatus as setStoreBackendStatus } from "./useDataStore";
|
||||
import { appendLiveDataBoundsParams } from "@/lib/liveDataViewport";
|
||||
|
||||
export type BackendStatus = 'connecting' | 'connected' | 'disconnected';
|
||||
|
||||
@@ -32,8 +33,8 @@ export async function forceRefreshLiveData(): Promise<void> {
|
||||
|
||||
try {
|
||||
const [fastRes, slowRes] = await Promise.all([
|
||||
fetch(`${API_BASE}/api/live-data/fast`),
|
||||
fetch(`${API_BASE}/api/live-data/slow`),
|
||||
fetch(appendLiveDataBoundsParams(`${API_BASE}/api/live-data/fast`)),
|
||||
fetch(appendLiveDataBoundsParams(`${API_BASE}/api/live-data/slow`)),
|
||||
]);
|
||||
|
||||
if (fastRes.ok) {
|
||||
@@ -85,9 +86,13 @@ export const LAYER_TOGGLE_EVENT = 'sb:layer-toggle';
|
||||
/**
|
||||
* Polls the backend for fast and slow data tiers.
|
||||
*
|
||||
* All data is fetched globally (no bbox filtering) — the backend returns its
|
||||
* full in-memory cache and MapLibre culls off-screen entities on the GPU.
|
||||
* This eliminates the "empty map when zooming out" lag.
|
||||
* Issue #288: heavy, density-driven layers (vessels, aircraft, gdelt
|
||||
* events, fires, sigint, …) are bbox-scoped to the visible map area via
|
||||
* `appendLiveDataBoundsParams`. Static reference layers (datacenters,
|
||||
* military bases, power plants, satellites, weather, news, …) are NOT
|
||||
* filtered backend-side, so panning never reveals an "empty world" of
|
||||
* infrastructure. World-zoomed views skip bbox params entirely and hit
|
||||
* the shared ETag cache exactly like the pre-#288 behaviour.
|
||||
*
|
||||
* The AIS stream viewport POST (/api/viewport) is still handled separately
|
||||
* by useViewportBounds to limit upstream AIS ingestion.
|
||||
@@ -147,7 +152,9 @@ export function useDataPolling() {
|
||||
const useStartupPayload = !fetchedStartupFastPayload && !fastEtag.current;
|
||||
const headers: Record<string, string> = {};
|
||||
if (!useStartupPayload && fastEtag.current) headers['If-None-Match'] = fastEtag.current;
|
||||
const url = `${API_BASE}/api/live-data/fast${useStartupPayload ? '?initial=1' : ''}`;
|
||||
const url = appendLiveDataBoundsParams(
|
||||
`${API_BASE}/api/live-data/fast${useStartupPayload ? '?initial=1' : ''}`,
|
||||
);
|
||||
const res = await fetch(url, {
|
||||
headers,
|
||||
signal: controller.signal,
|
||||
@@ -193,10 +200,13 @@ export function useDataPolling() {
|
||||
try {
|
||||
const headers: Record<string, string> = {};
|
||||
if (slowEtag.current) headers['If-None-Match'] = slowEtag.current;
|
||||
const res = await fetch(`${API_BASE}/api/live-data/slow`, {
|
||||
headers,
|
||||
signal: controller.signal,
|
||||
});
|
||||
const res = await fetch(
|
||||
appendLiveDataBoundsParams(`${API_BASE}/api/live-data/slow`),
|
||||
{
|
||||
headers,
|
||||
signal: controller.signal,
|
||||
},
|
||||
);
|
||||
if (res.status === 304) { scheduleNext('slow'); return; }
|
||||
if (res.ok) {
|
||||
slowEtag.current = res.headers.get('etag') || null;
|
||||
|
||||
Reference in New Issue
Block a user