Add Earthdata token entry on the SAR tab with accurate Mode B status, expose optional FIRMS_MAP_KEY in API settings, and remove the frontend operator-unlock gate that blocked localhost saves. Run network-heavy layer-enable fetches on a background executor with frontend retry polling so FIRMS toggles no longer freeze the single API worker.
Drop duplicate slow-tier weather/ukraine jobs, gate correlations when off, slim health probes, keyed layer-panel subscriptions, align backend layer defaults with the dashboard, and fetch CCTV/FIRMS/PSK/etc. synchronously on enable so toggles stay responsive without background prefetch waste.
Replace spoofable Host/Origin WebSocket auth with short-lived bootstrap
tokens minted over the existing local-operator HTTP path. Docker/browser
shell sessions prefetch a token before connecting; loopback peers remain
unchanged.
Also bump backend ws to 8.21.0 and refresh frontend lockfile to clear
npm audit findings (dev toolchain only for frontend).
Fixes#405, #406, #407
Co-authored-by: Cursor <cursoragent@cursor.com>
The five-tab settings bar overflowed the 480px drawer, so the Infonet
active underline painted past the right edge. Use a clipped grid layout
with truncated labels and min-h-0 scrolling on the Infonet tab.
Fixes#395
Co-authored-by: Cursor <cursoragent@cursor.com>
Unify toggle-all exclusions for Earth imagery overlays so the icon matches layer state, and let Docker operators save news feeds via the proxy without a misleading network error.
Auto-relay connect DMs with End Contact severing, signed fleet prekey lookup,
OpenClaw private Infonet channel intents, headless relay Tor bootstrap on redeploy,
and swarm/DM live verification scripts.
Co-authored-by: Cursor <cursoragent@cursor.com>
Ship sb-testnet fleet defaults, swarm/join API, NODE launcher registration step, and meshnode script defaults so users discover peers via the signed seed manifest without manual peer lists.
Co-authored-by: Cursor <cursoragent@cursor.com>
Signed peer manifest pull/announce on the seed, immediate hashchain push for gate messages, seed-only Docker defaults, and stale-genesis sync diagnostics.
Co-authored-by: Cursor <cursoragent@cursor.com>
Rename Mesh Chat to Meshtastic Chat, embed the Infonet terminal with Arti/Tor warmup, improve the agent shell PTY (git in the backend image, operator PATH), and add docker-compose.override for local image builds. Gitignore Hermes Agent runtime installs.
Co-authored-by: Cursor <cursoragent@cursor.com>
Uses a local-operator WebSocket bash session, keeps the map interactive, and SNAP docks the shell back into Mesh Chat instead of a floating blurred panel.
Co-authored-by: Cursor <cursoragent@cursor.com>
Anchors to the Mesh Chat box, stretches on tab enter, and supports user resize without changing the fixed left column width.
Co-authored-by: Cursor <cursoragent@cursor.com>
Align backend, desktop, helm, and frontend package versions for the Telegram OSINT and OpenClaw recon release.
Co-authored-by: Cursor <cursoragent@cursor.com>
Add Telegram OSINT with hourly incremental t.me scraping, metro geocoding
separate from news centroids, threat-intercept popup UI with inline media,
and HTML markers above alert boxes so pins stay clickable. Expose GFW_API_TOKEN
in onboarding and Settings Maritime; harden GFW/CCTV/geo fetchers. Port Osiris-
derived recon, SCM, entity graph, malware/cyber feeds, sanctions, and submarine
cable layers with tests and documentation.
Co-authored-by: Cursor <cursoragent@cursor.com>
Operators enable Polymarket/Kalshi correlation from Global Threat Intercept with a consent dialog; polls use a jittered schedule separate from the slow tier. Right-click Sentinel imagery returns up to three signed scenes again.
Co-authored-by: Cursor <cursoragent@cursor.com>
- User-Agent is per-install handle only (no Shadowbroker product token)
- LiveUAMap: Windows UI consent when enabling Global Incidents; env override
- Meshtastic callsign upstream header off by default (opt-in true)
- Expanded docs/OUTBOUND_DATA.md and README link for CCTV, basemap, Broadcastify
Co-authored-by: Cursor <cursoragent@cursor.com>
Private gate messages and offline DMs now ride the Infonet hashchain
as ciphertext-only events, replicated across nodes via private
transports (Tor onion / RNS / loopback) and decrypted only by parties
holding the gate or recipient keys.
Hashchain core (mesh_hashchain.py)
----------------------------------
* New ``append_private_gate_message`` and ``append_private_dm_message``
append paths with full signature verification, public-key binding,
revocation check, and replay protection in a dedicated sequence
domain (so a gate post does not consume the author's public broadcast
sequence, and a DM cannot replay-block a public message at sequence=1).
* Fork validation and full-chain validation now accept the gate
signature compatibility variants — older signatures that canonicalize
with/without epoch or reply_to still verify, so a re-sync from an
older peer doesn't reject still-valid history.
* DM hashchain spool: capped at 2 active sealed offline DMs per
recipient mailbox, plus a per-(sender, recipient) cap so one prolific
sender can't consume both slots. 1-hour TTL on the cap counter.
Spool intentionally small — it's an offline bootstrap channel,
not a persistent mailbox.
* Rebuild-state preserves the gate sequence domain across reloads so
a chain reload doesn't accidentally let an old gate sequence
replay-collide on next append.
Schema enforcement (mesh_schema.py)
-----------------------------------
* Private gate + DM payloads have closed allowlists of fields.
Plaintext keys (``message``, ``plaintext``, ``_local_plaintext``,
``_local_reply_to``) are explicit rejection-bait — they raise before
the event ever touches the chain.
* DM ciphertext + nonce must look like base64-ish sealed bytes;
obvious base64-encoded plaintext shapes are rejected.
* ``transport_lock`` required: DM hashchain spool requires
``private_strong``; gate accepts ``private``/``private_strong``/
``rns``/``onion``.
Defense-in-depth at the network layer (main.py + mesh_public.py)
----------------------------------------------------------------
* ``_infonet_sync_response_events`` now silently redacts private events
(gate_message + dm_message) unless the request looks like a loopback /
onion / RNS / private transport caller. If an operator accidentally
exposes :8000 to the public internet, an external puller gets
public events only — never ciphertext.
* ``_sync_from_peer`` raises ``PeerSyncRateLimited`` for 429 (handled
as 4-tuple return with retry_after_s) and ``PeerSyncHTTPError`` for
other non-200 statuses (handled by ``_run_public_sync_cycle`` to
honor server cooldown hints even outside the 429 path).
DM relay hydration (main.py)
-----------------------------
* New ``_hydrate_dm_relay_from_chain``: when accepted dm_message chain
events arrive on a node, they get deposited into the local DM relay
store with a deterministic sender_token_hash so re-sync of the same
event is idempotent. Recipients see the ciphertext as a normal DM
on their next poll and decrypt with their existing recipient key.
Other surfaces
--------------
* meshnode.bat / meshnode.sh now set ``MESH_INFONET_ALLOW_CLEARNET_SYNC=
false`` and the participant runtime flags by default so a freshly
spun-up node defaults to private-only sync.
* InfonetTerminal/InfonetShell.tsx adds a gate directory renderer for
the new private-gate workflow.
* docker-compose.relay.yml binds the relay backend to 127.0.0.1:8000
only; Tor's hidden service forwards onion traffic into 127.0.0.1.
Public clearnet :8000 stays off the network edge.
Tests
-----
* 7 new tests in test_private_gate_hashchain.py + test_private_dm_
hashchain.py covering: gate fork accepts ciphertext propagation,
gate fork rejects plaintext, append rejects plaintext before
normalize, append requires private_strong, append rejects
non-sealed ciphertext shape, DM spool 2-per-recipient + 1-per-pair
cap, DM hydration delivers to poll/claim.
* Updated test_mesh_node_bootstrap_runtime.py covers 429 backoff via
PeerSyncRateLimited 4-tuple AND PeerSyncHTTPError exception.
* Updated test_s14b_public_sync_gate_filter.py + test_s9b_gate_store_
hydration.py + test_gate_write_cutover.py cover the new private
redaction on public sync responses.
* test_private_gate_hashchain.py + test_private_dm_hashchain.py:
10 passed locally.
* Combined mesh-relevant suite (the 5 modified existing tests +
2 new): 17 passed.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
What this release does
----------------------
1. Establishes a fresh Tauri updater signing keypair. The previous keypair
(pubkey baked into v0.9.79 / v0.9.8) had no matching private key on
any maintainer-controlled machine — every prior release shipped
without signatures, so auto-update has never actually worked. v0.9.81
rotates to a new pubkey and ships signed installers + latest.json so
every release from here is a one-click upgrade.
2. Fixes the ``admin_session_required`` race in TopRightControls.tsx.
The updateAction state used to default to ``auto_apply`` at React-init
time. A click on the Update button before the async runtime probe
completed went down the auto_apply path (POST /api/system/update),
which throws ``admin_session_required`` on fresh sessions. Desktop
installs now default to ``manual_download`` based on synchronous
``window.__TAURI__`` detection at useState init.
One-time cost for current installs
----------------------------------
Anyone on v0.9.79 or v0.9.8 will see the in-app Update button still
trigger the broken path on their existing install (the fix only takes
effect once they're ON v0.9.81). The MANUAL DOWNLOAD button in the
update dialog opens the GitHub release page, where they grab the .msi
and run it. After that one manual hop, all future updates are seamless.
Release artifacts
-----------------
ShadowBroker_v0.9.81.zip 6.06 MB
42f8a51f9a5690d1e7349d90d8ecf2d163c9061d6cf90c69ee03647a785437ff
ShadowBroker_0.9.81_x64_en-US.msi 122.4 MB
a45b177c26c95d2b28d71592d7147e88ff4e104865f214fde11249d311ec9e25
ShadowBroker_0.9.81_x64-setup.exe 76.5 MB
eca884b9d37eeccd0f11c91dcc6f6ae1b3609d9dee72bd73c37c9a427babfef2
Plus .sig files for the .msi and .exe, plus a signed latest.json for
the Tauri updater endpoint.
Sizes match the v0.9.79 / v0.9.8 reference shape within drift for
the new TopRightControls patch.
release_digests.json keeps v0.9.79 + v0.9.8 blocks alongside v0.9.81
so operators still on those versions continue to validate cleanly
during the rollout transition.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Bumps every hardcoded 0.9.79 → 0.9.8 across backend, frontend,
desktop-shell, helm, lockfiles, test fixtures. Refreshes the in-app
ChangelogModal HEADLINE_FEATURES, NEW_FEATURES, and BUG_FIXES with the
v0.9.8 highlights.
Release artifacts built locally and hashed into release_digests.json:
ShadowBroker_v0.9.8.zip 6.06 MB
d506f6b8462ccb12096f0cd9462233be58928094240416b65fb3127bdd1f3820
ShadowBroker_0.9.8_x64_en-US.msi 122.4 MB
d4be4cb68c3e6409fff54c225acdcdd08e27d5d6d2b31616d78d2a4f6812991d
ShadowBroker_0.9.8_x64-setup.exe 76.5 MB
1115d1f5cf37edd03ea2c21d821c7626e1bf3319c990402aaa0293bca46fea67
Sizes match the v0.9.79 reference shape (5.76 MB / 117 MB / 72.9 MB)
within expected drift for new code. The .zip is a `git archive` of the
v0.9.8 source tree (matching v0.9.79's approach).
Audit confirms no .env, .key, .venv-dir, or cache files leaked into the
backend-runtime bundle. Python 3.11.9 + 199 site-packages + privacy_core
all staged correctly.
Headline changes since v0.9.79:
* Cumulative fuel/CO2 per flight (#317) — running totals since first
observation, not just per-hour rate.
* AIS maritime resilience (#314, #316) — outage banner + AISHub REST
fallback when AISStream WebSocket primary is offline.
* Data-layer repair (#311, #312) — UAP fallback respects the 60-day
cutoff; GPS jamming threshold tuning + nac_p=0 inclusion so the layer
actually fires.
* Per-flight source attribution (#313) — source field on every record.
* Cross-node DM mailbox replication (#309).
* Infonet sync HTTP 429 honored (#310).
Test fixtures updated:
* test_per_operator_outbound_attribution.py — added v0.9.8 UA strings
to the banned-aggregate-literals list (alongside v0.9.79).
* updateRuntime.test.ts — bumped asset filename fixtures to v0.9.8.
release_digests.json keeps the v0.9.79 block alongside v0.9.8 so
operators still on 0.9.79 validate cleanly during the rollout.
The accent narrowing fix in ChangelogModal (one feature uses 'purple',
two use 'cyan' so the renderer's `accent === 'purple'` comparison
still type-checks) is included.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Pre-fix the emissions tooltip only showed the per-hour *rate* — what most
users actually want is the cumulative *amount* burned. This adds running
totals computed by multiplying the model-based rate by the elapsed
observation time since we first saw the airframe.
New module ``flight_observations.py``:
* Tracks first_seen_at + last_seen_at per icao24 hex.
* Re-opens a fresh session when an aircraft is unseen for > 15 min
(treated as a new flight — landed and took off, or transited a dead
zone). Prevents the cumulative counter from resetting mid-flight if
the trail-rendering cache prunes the trail.
* Clamps elapsed time to 24h max so clock skew can't produce comically
large numbers.
* Pruned every 5 min via a new scheduler job (mirrors ais_prune cadence).
flights.py + military.py emission enrichment now also attaches:
* observed_seconds — how long we've been tracking this airframe.
* fuel_gallons_burned — rate * elapsed_h.
* co2_kg_emitted — rate * elapsed_h.
The existing per-hour rate fields stay in the dict for backward compat
and are shown as small secondary context in the tooltip.
Frontend EmissionsEstimateBlock (NewsFeed.tsx) now prominently shows
the cumulative totals with the rate as smaller context underneath plus
"Observed in flight for Xh Ym". When observed_seconds is 0 (first refresh)
it renders "Just observed · totals will appear on next refresh" instead
of a misleading "0 gal".
12 backend tests cover record/accumulate/reset, the 24h clamp, prune,
case-insensitive key normalization, and end-to-end emission integration
in _classify_and_publish.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
On 2026-05-23, stream.aisstream.io went fully offline (TCP timeouts on port
443). The backend kept respawning the node WebSocket proxy every few
seconds with nothing arriving. From the operator's POV the ships layer
silently went empty — no banner, no log surfacing, no way to tell whether
it was their config / network / viewport filter / upstream.
Backend:
* ais_proxy_status() now also returns:
- connected (bool): true when a vessel message arrived in last 60s
- last_msg_age_seconds (int | None)
- proxy_spawn_count (int): proxy respawns — sustained growth without
connected means upstream is dead
* /api/health escalates top status to "degraded" when AIS_API_KEY is set
but the proxy is currently disconnected. Existing degraded_tls signal
preserved.
Frontend:
* useAisUpstreamHealth hook polls /api/health every 30s, derives the
outage state. Defensively only reports outage once spawn_count > 0 so
operators who haven't opted in don't see the banner.
* AisUpstreamBanner component renders a dismissible amber notice
"Ship data temporarily unavailable — AISStream upstream is offline"
mounted on the main app shell.
7 backend tests pin the status-shape contract and the /api/health
escalation behavior in both with-key and without-key configurations.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reported by @tg12. Pre-fix, two problems lived on the GET endpoint:
1. `GET /api/ai/connect-info?reveal=true` returned the full HMAC
secret in the response body on every Connect modal open. Even
gated to require_local_operator, that put the secret into
browser history, dev-tools network panels, browser disk caches,
HAR exports, and screen captures.
2. The same GET endpoint auto-bootstrapped (generated + persisted)
the secret on a mere read. Side effects on a GET are a footgun:
browser prefetchers, mirror tools, and casual curl-from-history
would all silently mint+persist a fresh secret.
Backend (backend/routers/ai_intel.py)
-------------------------------------
GET /api/ai/connect-info — always returns the MASKED
fingerprint (first6 + bullets
+ last4). No `?reveal` param.
NO auto-bootstrap. When the
secret is missing, returns
`hmac_secret_set: false` and
tells the caller to POST to
/bootstrap.
POST /api/ai/connect-info/bootstrap — NEW. Mints+persists the secret
if missing. Idempotent. Never
returns the full secret in the
response body.
POST /api/ai/connect-info/reveal — NEW. Returns the full secret
with Cache-Control: no-store,
no-cache, must-revalidate +
Pragma: no-cache + Expires: 0.
POST so the body never lands
in URL history. 404 (with a
pointer to /bootstrap) when
the secret isn't set.
POST /api/ai/connect-info/regenerate — keeps existing one-time-reveal
behavior (regen IS a deliberate
destructive action triggered
by the operator). Same
no-store/no-cache headers added
so even the regen response
doesn't get cached.
Frontend (AIIntelPanel.tsx, OnboardingModal.tsx)
------------------------------------------------
* On mount: GET (masked only). If hmac_secret_set: false, fire a
transparent POST /bootstrap and refresh the masked fingerprint.
Operator sees no behavior change from pre-#302.
* Reveal (eye icon): lazy POST /reveal — secret only travels when
the operator explicitly clicks the button.
* Copy: lazy POST /reveal too — copying without a prior reveal
works exactly like before, just routed through the new endpoint.
* Regenerate: POST returns the new secret (same as before, but the
response now has no-store headers).
* The displayed snippet uses the masked fingerprint until the
operator clicks Reveal or Copy.
Tests (backend/tests/test_openclaw_connect_info_reveal.py — 13 tests)
---------------------------------------------------------------------
* GET returns masked + the full secret never appears in r.text
* GET does NOT auto-bootstrap when missing
* GET silently ignores any ?reveal=true query (back-compat noise)
* POST /bootstrap mints when missing, idempotent when set
* POST /bootstrap never returns the full secret
* POST /reveal returns the full secret with Cache-Control: no-store,
no-cache + Pragma: no-cache + Expires: 0
* POST /reveal 404s with a pointer to /bootstrap when no secret
* POST /regenerate returns the new secret with the same headers
* Anonymous remote callers get 403 on ALL FOUR endpoints (parametric
regression against the same allowlist used elsewhere).
Adjacent suites still green: test_openclaw_route_security,
test_no_new_duplicate_routes, test_control_surface_auth. 67/67 pass
locally.
Credit: @tg12 for the audit report.
Reported by @tg12. Pre-fix, the Settings panel stored real third-party
Copernicus CDSE client_id + client_secret in browser localStorage /
sessionStorage via the privacy storage helper, and the proxy routes
required those values to come back in every tile/token request body.
Any same-origin script (XSS, malicious browser extension, dev-tools
HAR export) had read access to the credentials.
This change moves them server-side, behind the same .env-backed admin
flow every other third-party API key (OpenSky, AIS Stream, Finnhub,
Shodan, …) already uses.
Backend
-------
backend/services/api_settings.py
* Added SENTINEL_CLIENT_ID and SENTINEL_CLIENT_SECRET entries to
API_REGISTRY. The existing GET/PUT /api/settings/api-keys flow
(already require_local_operator-gated, .env-backed) now manages
them — no new route surface.
backend/routers/tools.py
* /api/sentinel/token and /api/sentinel/tile resolve credentials via
a new _resolve_sentinel_credentials() helper: body fields win for
back-compat with any legacy callers, otherwise the helper reads
SENTINEL_CLIENT_ID / SENTINEL_CLIENT_SECRET from os.environ.
* When neither source has a value, the route returns 400 with a
friendly pointer ("Set SENTINEL_CLIENT_ID and SENTINEL_CLIENT_SECRET
in the API Keys panel") instead of the curt "required" message.
The user's standing rule against hostile errors applies.
* Function bodies only — decorator lines untouched, so this PR does
not conflict with #303 (which adds Depends(require_local_operator)
to the same routes).
Frontend
--------
frontend/src/lib/sentinelHub.ts — rewritten
* Removed: getSentinelCredentials / setSentinelCredentials /
clearSentinelCredentials / getSentinelCredentialStorageMode.
These were the browser-storage read/write helpers; their existence
was the bug.
* Added: checkBackendSentinelStatus(), refreshSentinelStatus(),
getCachedSentinelStatus(), and a kept-for-back-compat
hasSentinelCredentials() shim. Status is sourced from
/api/settings/api-keys (the same endpoint the API Keys panel
already uses), so we don't add a new route just for this read.
* Added: migrateLegacySentinelBrowserKeys() — one-shot, idempotent
helper that clears sb_sentinel_client_id / _secret / _instance_id
from BOTH localStorage and sessionStorage. We deliberately do NOT
auto-POST those legacy browser values to the backend; doing so
would silently migrate a secret across a trust boundary without
operator consent. Operators re-enter once in the API Keys panel
and the legacy keys get wiped here.
* fetchSentinelTile and getSentinelToken no longer send client_id /
client_secret in the request body. The backend uses .env.
frontend/src/components/SettingsPanel.tsx
* Dropped sb_sentinel_client_id / _secret / _instance_id from
PRIVACY_SENSITIVE_BROWSER_KEYS — they're no longer written.
* SentinelTab rewritten: removed the inline Client ID / Client Secret
inputs + Save / Clear / Test buttons. Replaced with a status panel
that calls checkBackendSentinelStatus() on mount, a one-click
"Open API Keys Panel" button, and a migration banner that appears
only when migrateLegacySentinelBrowserKeys() actually cleared
something.
* Setup guide STEP 3 now points to the API Keys panel instead of
the local form.
frontend/src/app/page.tsx
* Added a one-time useEffect that fires checkBackendSentinelStatus()
on mount so the cached value (which the synchronous
hasSentinelCredentials() shim reads) is populated before
MaplibreViewer's tile-URL memo runs.
Tests
-----
backend/tests/test_sentinel_credentials_server_side.py (new)
* API_REGISTRY surface — sentinel_client_id / sentinel_client_secret
are registered with the right env_keys, ALLOWED_ENV_KEYS lets
/api/settings/api-keys PUT them.
* Resolution order — body wins, env is fallback, neither → 400 with
the friendly pointer message, and NO upstream HTTP call when
neither source has credentials (asserted via
MagicMock(side_effect=AssertionError)).
* /api/sentinel/tile same shape.
frontend/src/__tests__/utils/sentinelHub.test.ts (new)
* migrateLegacySentinelBrowserKeys clears localStorage AND
sessionStorage, reports what it cleared, idempotent.
* fetchSentinelTile + getSentinelToken POST WITHOUT client_id /
client_secret in the body (plants leaked credentials in browser
storage first to prove they are NOT picked up).
* checkBackendSentinelStatus parses /api/settings/api-keys correctly:
true only when both keys is_set, false on partial config or
network errors.
All 7 backend tests + 8 frontend tests pass locally. The
test_no_new_duplicate_routes guard and the api-settings test suite
still pass.
Credit: @tg12 for the audit report.
Reported by @tg12 in the external security/correctness audit.
Before this change, /api/live-data/{fast,slow} accepted s/w/n/e query
params but their Query() descriptions explicitly said "(ignored)". The
endpoints shipped the full in-memory world dataset on every poll:
/api/live-data/fast → 16.88 MB
/api/live-data/slow → 10.12 MB
── 27 MB per poll cycle, regardless of zoom
For a node with N operators each polling at the steady 15s/120s cadence,
this is hundreds of MB/minute of outbound traffic that never gets used —
the GPU just culls everything outside the viewport client-side. On a
Tor-bridged or LTE-backed node, that bandwidth bill is the actual cost.
This change makes the existing s/w/n/e params honored — when all four
bounds are supplied, the backend bbox-filters a curated set of heavy,
density-driven, time-sensitive collections to that viewport (with the
existing 20% padding from _bbox_filter):
/fast: commercial_flights, military_flights, private_flights,
private_jets, tracked_flights, ships, cctv, uavs, liveuamap,
gps_jamming, sigint, trains
/slow: gdelt, firms_fires, kiwisdr, scanners, psk_reporter
Static reference layers (satellites, datacenters, military_bases,
power_plants, satnogs, weather, news, stocks, etc.) deliberately STAY
world-scale so panning never reveals an "empty world" of infrastructure.
That preserves the no-hostile-UX feel of the existing dashboard.
Behavior contract:
* Without bbox params (or with a partial bbox), the response is
byte-for-byte identical to the pre-#288 implementation. No
behavior change for any existing caller that hasn't opted in.
* World-scale bbox (lng_span >= 300 or lat_span >= 120) short-circuits
filtering and shares the global ETag — zoomed-out operators all
hit the same 304 cache exactly like before.
* ETag now mixes a 1°-quantized bbox suffix when filtering engages,
so two viewports never poison each other's 304 cache. Sub-degree
pans land in the same ETag bucket (i.e. don't bust the cache on
every mouse drag).
Polling cadence, rate-limit windows, and the 304 short-circuit are all
unchanged. Only the SIZE of the responses changes, and only when the
caller opts in via bounds.
Frontend wiring: useViewportBounds reuses the same coarsened/
expanded bounds it already computes for the AIS /api/viewport POST and
pushes them into a new module-level liveDataViewport store.
useDataPolling reads from that store via appendLiveDataBoundsParams
when building each live-data URL.
Tests cover: no-bbox → world data; bbox → heavy layers filtered;
bbox → reference layers untouched; world-scale bbox → no filter;
partial bbox → treated as no bbox; ETag changes with bbox; sub-degree
pan → same ETag; 304 path works; antimeridian-crossing bbox handled.
Co-authored-by: BigBodyCobain <moatbc@gmail.com>
Background
==========
PR #285 set up the seed -> cache -> GDELT model for the carrier tracker
to address audit issues #244/#245/#246. The GDELT half of that pipeline
hits api.gdeltproject.org's doc API for headline-region keyword
matching -- low precision (false centroid positions per #245) AND
unreliable (the host times out from some networks, including Docker
Desktop on Windows).
USNI publishes a weekly Fleet & Marine Tracker with explicit prose like:
"The Gerald R. Ford Carrier Strike Group is operating in the Red Sea"
"Aircraft carrier USS George Washington (CVN-73) is in port in
Yokosuka, Japan"
That is a strictly better source for U.S. Navy carrier positions:
authoritative, deterministically parseable, weekly cadence.
What this PR does
=================
New module: backend/services/fetchers/usni_fleet_tracker.py
- Pulls USNI's WordPress RSS feeds (site-wide + category, unioned).
- Picks the most recent fleet-tracker post by parsed pubDate.
- For each carrier in the registry, scans the article body for
"is operating in / is in port in / returned to / transiting" near
the carrier's name, hull code, or "<name> Carrier Strike Group"
variant. Captures the region/port phrase that follows.
- Maps the region phrase to coordinates via the existing
REGION_COORDS table, with a USNI-phrase alias table for the
specific wording USNI uses ("Yokosuka, Japan", "Norfolk, Va.",
"Naval Station San Diego", "5th Fleet AOR", etc.).
- Returns {hull: position_entry} with position_confidence="recent"
and position_source_at = the article's actual publication
timestamp (not now()).
Politeness
----------
Uses outbound_user_agent("usni-fleet-tracker") so USNI sees a
per-install Shadowbroker identifier (Round 7a / PR #292). The
article body pages return 403 to non-browser UAs; the WordPress RSS
feed serves the full <content:encoded> body and is the supported
aggregator path. No browser UA spoofing.
carrier_tracker.update_carrier_positions() now runs three phases:
1. Bootstrap from cache (or seed on first run).
2. USNI fleet tracker -- PRIMARY high-confidence source.
3. GDELT -- SECONDARY backfill; can NOT demote a "recent" USNI
position to an "approximate" GDELT headline match.
Verified live: 6 of 11 carriers picked up real May 18, 2026 positions
on first refresh (Eisenhower, Ford, Bush, Roosevelt, Lincoln,
Washington). The other 5 weren't mentioned in this week's article
(they're in port at homeports with no deployment changes) and kept
their cache entries -- which is the correct seed/cache contract from
PR #285.
Other small fixes bundled in
============================
docker-compose.yml: add the 6 third-party-fetcher opt-in env vars
(PREDICTION_MARKETS_ENABLED, FINANCIAL_ENABLED, FIMI_ENABLED,
NUFORC_ENABLED, NEWS_ENABLED, CROWDTHREAT_ENABLED). They were
documented in .env.example but never wired through compose, so setting
them in .env had no effect.
frontend/src/components/TopRightControls.tsx: fix 6 broken i18n keys
that were showing as raw "terminal.term1" / "terminal.cleanupDetail" /
"node.soloReady" placeholders in the INFONET TERMINAL modal. The
translation files have these strings under different key names; the
component now calls the right ones. Full-file sweep confirmed every
other t('...') key in the whole frontend resolves cleanly.
Wikimedia's User-Agent policy asks API clients to identify themselves
with a stable, contactable identifier so their operators can rate-limit
or coordinate. Before this change, ShadowBroker was sending:
- Backend (region_dossier.py): generic project default UA only; no
Api-User-Agent.
- Frontend (useRegionDossier.ts, WikiImage.tsx, NewsFeed.tsx): zero
identifying header at all; three separate copy-pasted anonymous
fetches with their own module-local caches.
Three separate components doing the same broken thing meant policy
fixes had to happen in three places, with no shared cache or kill
switch.
Fix (no UX change, zero hostility):
== Backend ==
`backend/services/region_dossier.py` now sets explicit `User-Agent` +
`Api-User-Agent` headers on every outbound Wikidata and Wikipedia
request via a new `_WIKIMEDIA_REQUEST_HEADERS` constant. The identifier
includes a contact path (issues page on the public GitHub repo).
== Frontend ==
New shared helper `frontend/src/lib/wikimediaClient.ts`:
- `fetchWikipediaSummary(title)` — single source of truth for Wikipedia
REST summary lookups, with one shared LRU cache (in-flight requests
deduplicated, 512-entry cap), `Api-User-Agent` on every fetch.
- `fetchWikidataSparql(query)` — same shape for Wikidata SPARQL.
- `WIKIMEDIA_API_USER_AGENT` — exported constant; one place to update
if Wikimedia ever asks us to back off.
Refactored three components to use the shared client:
- `frontend/src/hooks/useRegionDossier.ts` — fetchLeader() and
fetchLocalWikiSummary() now route through the shared helpers.
- `frontend/src/components/WikiImage.tsx` — uses fetchWikipediaSummary,
proper React state instead of module-mutation + forceUpdate trick.
- `frontend/src/components/NewsFeed.tsx` — same shape.
UX: byte-for-byte identical. Same thumbnails, same dossier content,
same load behavior. The only observable difference is the outgoing
request header.
Note on #239 (route duplication): an audit-grade inventory shows 166
main.py routes are shadowed by router modules. That cleanup is too
large to land safely in this PR; it will be staged as a separate
ladder of small PRs grouped by router module.
Tests:
- `backend/tests/test_region_dossier_wikimedia_ua.py` — 3 tests
asserting backend headers are present.
- `frontend/src/__tests__/utils/wikimediaClient.test.ts` — 9 tests
covering Api-User-Agent presence, shared cache, concurrent
deduplication, disambiguation/HTTP-error/network-error fallthroughs,
empty-input safety.
Local: backend 76/76 security suite green, frontend 716/716 vitest
suite green.
Credit: tg12 (external security audit).
PR #226 landed the i18n infrastructure and Chinese (zh-CN) translations.
This follow-up adds the safeguards that make accepting community
translations sustainable without exposing the project to subtle
state-aligned framing in future translation PRs.
Changes:
frontend/src/i18n/index.tsx (renamed from .ts)
- Add LOCALES registry: a single source of truth for available
languages and their NATIVE display names ("English", "中文 (简体)").
Adding a new language is now a one-entry change here plus a
JSON file.
- Add isLocale() guard so an unknown value in localStorage falls
through to navigator.language detection instead of corrupting
state.
- File renamed to .tsx because it contains JSX. Next.js tolerated
JSX in .ts but Vite/Oxc (used by vitest) does not.
frontend/src/components/SettingsPanel.tsx
Add a UI language picker to the Settings header — a small <select>
populated from LOCALES. Users no longer need the dev console to
switch languages. Locale change remains 100% client-side
(localStorage), no network call, no telemetry.
CONTRIBUTING.md (new)
Documents the translation-neutrality requirement that applies
symmetrically to all source countries:
- Translations must be technically faithful to the English source.
- Substitutions aligned with state propaganda from ANY country
(PRC, Russia, US, EU, etc.) will be rejected.
- The test is: "would a translator working strictly from the
English source produce this rendering?"
Also explains how translation PRs are reviewed and how to add
a new language.
.github/CODEOWNERS (new)
Auto-requests maintainer review on:
- /frontend/src/i18n/ (translation safety)
- /backend/auth.py, /backend/routers/wormhole.py,
/backend/services/mesh/, /backend/services/fetchers/
(the same paths recent security audits flagged as sensitive)
- /.github/workflows/, /.gitlab-ci.yml, /docker-compose*.yml,
/helm/ (build/deploy)
- /CONTRIBUTING.md, /.github/CODEOWNERS (policy itself)
frontend/src/__tests__/i18n/i18nProvider.test.tsx (new, 8 tests)
Locks in the i18n contract:
- LOCALES has both en and zh-CN with non-empty native labels
- Default English when navigator is English
- Auto-detect zh-CN when navigator language starts with "zh"
- localStorage preference overrides auto-detect
- setLocale persists to localStorage
- Unknown stored locale falls back to auto-detect
- Renders a real zh-CN translation (catches large-scale
translation removal in future PRs)
- Missing key falls back to the key itself
Note: i18n/index.tsx, the language toggle UI, the translation
policy, and the test suite together form a defense-in-depth setup.
The structural safety guarantee (no network calls, static JSON
bundled at build) is intact; this PR makes the social contract
around translations explicit and enforceable via branch
protection on CODEOWNERS-marked paths.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Introduce a lightweight i18n system with auto-detection of browser
language and localStorage persistence. Add complete Chinese translations
for all major UI sections: navigation, controls, update dialogs, node
activation, terminal launcher, data layers, settings, filters, and more.
Technical terms (Wormhole, Infonet, Mesh, Shodan, SAR, etc.) are
intentionally kept in English. Falls back to English when Chinese
translation is not found.
Co-authored-by: wangsudong <wangsudong@kylinos.cn>
Each alert toast had a 5-second auto-dismiss timer that fired even
while the user was reading the card. This adds pause-on-hover: the
dismiss timer stops while the mouse is over a toast and restarts (full
lifetime) on mouse leave. The progress bar animation pauses with it,
so the visual matches the actual remaining time.
All other behavior is preserved: same cyber/mono styling, same spring
slide-in, same risk-color border + glow, same warning icon, same
LVL X/10 readout, same title/source layout, same click-to-fly + dismiss
on body click, same × dismiss button.
Implementation notes:
- Extract a ToastCard sub-component so each card can own its own
paused state (useState can't be array-indexed in the parent).
- Move the auto-dismiss timer out of useAlertToasts.ts and into
ToastCard. The hook previously scheduled the dismiss itself, which
meant the UI couldn't pause it — only the component knows whether
the user is interacting.
- Add tests covering: title/source/severity render, auto-dismiss
fires at 5s, hover pauses indefinitely, mouse-leave restarts the
full lifetime, × dismisses without flying, body-click flies +
dismisses.
This implements the genuine UX improvement that PR #234 was reaching
for, without #234's broken syntax, missing-field bug, duplicate
timer logic, or design regression.
Refs: #234
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Allow local-operator DM invite import without requiring a full admin session.
Prioritize bundled/bootstrap seed peers and shorten stale seed cooldowns for faster Infonet recovery.
Replace raw DM invite dumps with copyable signed-address controls, contact request handling, and safer sealed-send behavior while the private delivery route connects.
Ship the v0.9.79 runtime refresh with transport lane isolation, Infonet secure-message address management, MeshChat MQTT controls, selected asset trail behavior, telemetry panel refinements, onboarding updates, and desktop/package metadata alignment.
Also ignore local graphify work products so analysis folders do not leak into future commits.
Add Tor/onion runtime wiring and faster Infonet node status refresh.
Keep node bootstrap state clearer across Docker and local runtimes.
Use selected aircraft trail history for cumulative tracked-aircraft emissions.
Let fresh Docker and local installs enter OpenSky, AIS, and other provider keys directly in onboarding or Settings without manually creating .env files. Persist keys server-side in the backend data store, keep them write-only from the browser, reload runtime settings, and retain local-operator access controls.
Allow the bundled Docker frontend proxy to reach local-operator endpoints through the private compose bridge without trusting LAN clients. This restores Time Machine, MeshChat key creation, AI pins/layers, and related local controls in Docker installs. Refresh first-run guidance so Docker users know to configure OpenSky and AIS keys through .env.
BigBodyCobain