Detected by Aeon + Semgrep (5x use-defused-xml ERROR).
Severity: medium
CWE-776 (billion laughs) / CWE-611 (XML external entity)
Five XML parse sites pass response bodies into the Python stdlib
xml.etree.ElementTree without protection against entity expansion
attacks. Python's ElementTree still permits internal entity references
by default (per the docs vulnerabilities table), so a malicious or
compromised upstream can ship a "billion laughs"-style payload that
expands to gigabytes in memory.
The user-controllable site is sb_monitor._parse_rss: the OpenClaw skill
exposes add_custom_feed(name, url, ...) to the agent, then
poll_custom_feeds fetches feed.url and passes the body to
xml.etree.ElementTree.fromstring with no host allowlist or
entity-bomb defence. The other four sites (psk_reporter_fetcher,
aircraft_database, cctv_pipeline x2) parse XML from hard-coded
upstreams (pskreporter.info, s3.opensky-network.org,
datos.madrid.es); defence-in-depth for upstream-compromise/MITM.
Switch all five call sites to defusedxml.ElementTree. Same
fromstring/find/findall/iter/findtext API, but rejects entity
references by default (raises defusedxml.EntitiesForbidden).
Confirmed locally that a 4-deep billion-laughs payload that
expands to 3000 chars under stdlib ET is rejected by defusedxml.
Added defusedxml>=0.7.1 to backend/pyproject.toml dependencies.
Co-authored-by: aeonframework <aeon-bot@aaronjmars.com>
External security audit by @tg12 (May 17, 2026) filed issues #201–#214
in addition to the #189–#200 batch already closed by PRs #227/#232/#260.
This PR closes all eight that are real security bugs (the other six in
the 201–214 range are either design discussions or upstream-abuse/TOS
concerns we're keeping intentional, see issue triage notes on each).
The user-facing principle for this PR: fix the security gap WITHOUT
introducing a single hostile error or behavior change for legitimate
users. Every fix follows the same template — fail forward, not loud.
When the secure path is harder than the insecure one, build a
fallback chain that ends in graceful degradation, not in a scary
modal or 422 response.
#205 — OpenMHZ audio redirect SSRF (services/radio_intercept.py)
Replaced requests.get(..., allow_redirects=True) with a manual
redirect loop that re-validates each hop's host against
_OPENMHZ_AUDIO_HOSTS. Same-host redirects (CDN edge selection)
still work, so legitimate audio playback is unaffected. Cross-host
redirects to disallowed hosts return a generic 502 which the
browser audio element handles gracefully. Cap at 5 hops.
#207 — infonet/status verify_signatures DoS (routers/mesh_public.py)
Silently downgrade verify_signatures=true to False for
unauthenticated callers. No error surfaced — the response shape is
identical, just without the O(n_events) signature verification.
Authenticated callers (scoped mesh.audit) still get the full path.
The frontend never passes this param so legitimate UI is unaffected.
#211 — thermal/verify expensive analysis (routers/sigint.py)
Added Depends(require_local_operator). Frontend has no direct
callers (verified by grep); Tauri/AI agents use scoped tokens that
pass the auth check. Anonymous abusers blocked silently — the
legitimate UI keeps working through the Next.js admin-key proxy.
#213, #214 — OpenMHZ calls/audio upstream abuse (routers/radio.py)
Added Depends(require_local_operator) to both. Browser users hit
these through the Next.js proxy at src/app/api/[...path]/route.ts
which injects X-Admin-Key, so the auth check passes transparently.
Direct attackers can no longer rotate sys_names to hammer
api.openmhz.com or relay arbitrary audio streams through the
backend's bandwidth.
#202 — overflights unbounded hours (routers/data.py)
Silently clamp `hours` to OVERFLIGHTS_MAX_HOURS (default 72,
configurable). NO 422 — clients asking for an absurd window get a
shorter window back with `requested_hours` and `effective_hours`
hint fields. Postel's law: liberal in what we accept, conservative
in what we compute.
#203 — Meshtastic callsign UA leak (services/fetchers/meshtastic_map.py)
Added MESHTASTIC_SEND_CALLSIGN_HEADER opt-out env var. Default is
TRUE — preserves existing operator behavior (callsign sent so
meshtastic.org can rate-limit per-install). Privacy-conscious
operators set it to false to suppress.
#206 — KiwiSDR upstream is HTTP-only (services/kiwisdr_fetcher.py)
Upstream rx.linkfanel.net doesn't speak HTTPS (verified — Apache
2.4.10 only on port 80). We can't fix the transport. Instead added
three layers:
1. Content validation on fetched data — reject responses with
<50 receivers or >5% malformed entries (likely MITM injection).
2. Existing disk cache fallback (already present).
3. NEW: bundled static directory at backend/data/kiwisdr_directory.json
shipping 798 known-good receivers. Used as last resort so the
KiwiSDR map layer always renders something useful.
#208 — Merkle proof DoS via /api/mesh/infonet/sync (services/mesh/mesh_hashchain.py)
The endpoint is part of the cross-node federation protocol — peers
legitimately call it without local-operator auth, so we can't add
Depends(). Instead made the underlying operation O(1) per proof
via a cached Merkle level structure on the Infonet instance:
- _merkle_levels_cache + _merkle_levels_for_event_count on each
Infonet instance
- _invalidate_merkle_cache() called from every chain mutation
point (append, ingest_events, apply_fork, cleanup_expired)
- _get_merkle_levels() does the lazy recompute on first read
after invalidation, then serves from cache thereafter
Effect: anonymous attackers hammering the proofs endpoint hit a
cached structure; the rebuild happens at most once per real chain
advance. Federation untouched.
#201 — Tor bundle SHA-256 bypass (services/tor_hidden_service.py)
Docker users were already covered — backend/Dockerfile installs
Tor via apt-get at build time (signed by Debian's package system).
No runtime download needed for the 80%-of-users case.
For Tauri desktop, replaced the single .sha256sum check with a
multi-source verification chain implemented in _verify_tor_bundle():
1. Try upstream .sha256sum (current behavior — fast path)
2. Try baked-in digest list at backend/data/tor_bundle_digests.json
(pinned per-version, maintainer-updated)
3. If neither source is REACHABLE: HTTPS-only fallback with a loud
warning (avoids breaking first-run onboarding while the
maintainer hasn't yet pinned a new Tor release)
A mismatch from a source that DID respond is always fatal — only
the "no source reachable" case falls back to HTTPS-only. This is
the "have cake and eat it" pattern: real users see no new failure
modes during torproject.org outages, but MITM/compromise attacks
still fail because the downloaded digest can't match what BOTH
the upstream and the baked-in list report.
Currently the digest file ships with placeholder values for the
current Tor URLs (those URLs are already stale on torproject.org
too). A follow-up commit can populate real digests when a stable
Tor release is selected; until then the HTTPS-only warning fires
and onboarding still works.
Tests (82 total, all passing):
test_openmhz_redirect_ssrf.py (5 tests) — #205
test_infonet_status_verify_gate.py (2 tests) — #207
test_overflights_clamp.py (5 tests) — #202
test_meshtastic_callsign_optout.py (3 tests) — #203
test_kiwisdr_fallback.py (6 tests) — #206
test_merkle_cache.py (6 tests) — #208
test_tor_bundle_verification.py (6 tests) — #201
test_control_surface_auth.py (extended) — #211, #213, #214
+ all previous security tests (CCTV redirect, GDELT https, sentinel
cache, crowdthreat opt-in, third-party fetcher gates, control
surface auth) continue to pass.
Pre-existing test infrastructure issue with SHARED_EXECUTOR teardown
in the broader sweep exists on main too (verified) — not introduced
by this PR.
Credit: @tg12 reported every one of these with accurate line citations
and the recommended fixes that informed this implementation.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
External security audit by @tg12 (May 17, 2026) filed 11 issues against
the backend. PR #227 (May 18, AI-generated) closed seven of them by
adding require_local_operator to control-plane endpoints. Four remained
live; this PR closes the rest.
#192 — CCTV proxy followed redirects without re-validating host
Issue: /api/cctv/media validated only the caller-supplied URL host
before passing it to requests.get(..., allow_redirects=True). A 302
to http://127.0.0.1 or any internal/disallowed host was silently
followed, turning the proxy into an open-redirect-to-SSRF chain.
Fix in routers/cctv.py: replace the single allow_redirects=True call
with a manual follow loop. Each hop's Location is parsed, the host is
rerun through _cctv_host_allowed(), and non-HTTP schemes (file://,
ftp://, etc.) are rejected. Cap chain length at 5 hops.
Test: backend/tests/test_cctv_redirect_ssrf.py covers
- redirect to disallowed host -> 502
- redirect to localhost -> 502
- redirect to another allowed host -> 200
- redirect chain length cap
- non-HTTP scheme rejected
#198 — Gate introspection GETs were unauthenticated
Issue: /api/wormhole/gate/{gate_id}/{identity,personas,key} were
callable with no auth dependency. Any caller that could reach the
backend could dump the operator's active persona, persona inventory,
and key status for any gate_id they knew. The wiki's privacy threat
model explicitly markets gate personas as rotating, unlinkable
pseudonyms — this leak defeated that property.
Fix in routers/wormhole.py: add
dependencies=[Depends(require_local_operator)] to all three routes.
Test: backend/tests/test_control_surface_auth.py extended with
three new parameterized cases (lines 75-77).
#199 — GDELT military incident ingestion used plaintext HTTP
Issue: backend/services/geopolitics.py fetched
http://data.gdeltproject.org/gdeltv2/lastupdate.txt and ~48 export
archive URLs over plaintext HTTP. Passive observers could identify
Shadowbroker nodes from the fetch pattern. Active MITM could inject
doctored military incident records into the global map.
Fix in services/geopolitics.py: rewrite the lastupdate.txt fetch and
the export download URL constructor to use https://. GDELT's
data.gdeltproject.org serves the same content over HTTPS.
Test: backend/tests/test_gdelt_https.py asserts no plaintext HTTP
URLs to data.gdeltproject.org remain in code (comments excluded) and
that the HTTPS URLs we expect are present.
#200 — Sentinel token cache lookup used client_id only
Issue: routers/tools.py kept a process-global cache of Copernicus
bearer tokens. The lookup compared
_sh_token_cache["client_id"] == client_id. A caller who knew a valid
client_id but supplied any wrong client_secret hit the cache and
reused the legitimate caller's bearer token — burning their quota
and accessing imagery on their account.
Fix in routers/tools.py: replace the client_id field with
credential_fp, an HMAC-SHA256 over (client_id, client_secret) under
a per-process random key (_SH_TOKEN_CACHE_HMAC_KEY = os.urandom(32),
regenerated at startup). A caller who doesn't know the secret cannot
compute a matching fingerprint, so they miss the cache and hit the
real Copernicus token endpoint — which will reject their wrong
secret with a 401.
Test: backend/tests/test_sentinel_token_cache.py covers
- same client_id + different secrets => different fingerprints
- same credentials => same fingerprint (cache still works)
- different client_ids + same secret => different fingerprints
- cache no longer stores raw client_id (catches regression)
- attacker with wrong secret cannot reuse victim's token
Validation
pytest backend/tests/test_control_surface_auth.py
backend/tests/test_cctv_redirect_ssrf.py
backend/tests/test_gdelt_https.py
backend/tests/test_sentinel_token_cache.py
-> 37 passed
Credit: @tg12 reported all four of these in their May 17 audit with
correct line-number citations and accurate remediation recommendations.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
PR #227 hardened most Wormhole/Infonet control surfaces behind
require_local_operator and made the CrowdThreat fetcher opt-in. An
audit of the codebase against that PR's stated goals turned up four
classes of gap that the original change missed:
1. Two operator-only endpoints were left unprotected:
- POST /api/wormhole/join: calls bootstrap_wormhole_identity() and
flips the node into Tor mode, exactly the surface #227 hardened
on /api/wormhole/identity/bootstrap.
- POST /api/sigint/transmit: relays APRS-IS packets over radio
using operator-supplied credentials. Anything that reached the
API could transmit on the operator's authority.
Both now require_local_operator. test_control_surface_auth.py
extended with regression coverage for both.
2. Five third-party fetchers were still default-on, phoning home to
politically/commercially sensitive upstreams on every poll cycle:
- fimi.py -> euvsdisinfo.eu -> FIMI_ENABLED
- prediction_markets -> Polymarket + Kalshi -> PREDICTION_MARKETS_ENABLED
- financial.py -> Finnhub / yfinance -> FINANCIAL_ENABLED or FINNHUB_API_KEY
- nuforc_enrichment -> huggingface.co -> NUFORC_ENABLED
- news.py -> configured RSS feeds -> NEWS_ENABLED (default on, kill switch)
Same CrowdThreat-style pattern: explicit env-var opt-in, empty
the data slot and mark_fresh when disabled. New regression test
file test_third_party_fetchers_opt_in.py asserts each fetcher's
network entry point is not called when its gate is off.
3. The outbound User-Agent leaked both the operator's personal email
and a fork-specific GitHub URL on every fetcher request. Consolidated
to a single DEFAULT_USER_AGENT in network_utils.py, project-generic
by default (no contact info), overridable via SHADOWBROKER_USER_AGENT
for operators who want to identify themselves (e.g. for Nominatim or
weather.gov usage-policy compliance). Six call sites updated; the
Nominatim-specific override is preserved.
4. The same generic UA now also flows through the peer prekey lookup
in mesh_wormhole_prekey.py, so DM first-contact requests no longer
identify the caller as a Shadowbroker fork to the peer being
queried.
.env.example updated to document all new opt-in env vars.
Tests: backend/tests/test_control_surface_auth.py (extended),
backend/tests/test_crowdthreat_opt_in.py (unchanged, still passes),
backend/tests/test_third_party_fetchers_opt_in.py (new, 7 tests).
All 31 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Allow local-operator DM invite import without requiring a full admin session.
Prioritize bundled/bootstrap seed peers and shorten stale seed cooldowns for faster Infonet recovery.
Replace raw DM invite dumps with copyable signed-address controls, contact request handling, and safer sealed-send behavior while the private delivery route connects.
Ship the v0.9.79 runtime refresh with transport lane isolation, Infonet secure-message address management, MeshChat MQTT controls, selected asset trail behavior, telemetry panel refinements, onboarding updates, and desktop/package metadata alignment.
Also ignore local graphify work products so analysis folders do not leak into future commits.
Add Tor/onion runtime wiring and faster Infonet node status refresh.
Keep node bootstrap state clearer across Docker and local runtimes.
Use selected aircraft trail history for cumulative tracked-aircraft emissions.
Reduce cold-start stalls by raising the default backend memory limit, bounding heavy feed concurrency, preserving non-empty startup caches, and refreshing working news feeds. Fix the Next API proxy for Docker control-plane writes by stripping unsupported hop/body headers and forwarding small request bodies safely. Keep the dashboard dynamic so production users do not get stuck on a cached startup shell.
Let fresh Docker and local installs enter OpenSky, AIS, and other provider keys directly in onboarding or Settings without manually creating .env files. Persist keys server-side in the backend data store, keep them write-only from the browser, reload runtime settings, and retain local-operator access controls.
Allow the bundled Docker frontend proxy to reach local-operator endpoints through the private compose bridge without trusting LAN clients. This restores Time Machine, MeshChat key creation, AI pins/layers, and related local controls in Docker installs. Refresh first-run guidance so Docker users know to configure OpenSky and AIS keys through .env.
Seed safe static backend data into fresh Docker volumes, tighten Docker build-context exclusions, avoid optional env warnings, and make the frontend healthcheck use the IPv4 loopback path that works inside the container.
orjson ships pre-built wheels with AVX2 SIMD instructions that cause
SIGILL (exit code 132) on older processors. This wraps the import in
a try/except and falls back to stdlib json for serialization.
Closes#127
- SSE broadcast now uses loop.call_soon_threadsafe() when called from
background threads (gate pull/push loops), fixing silent notification
failures for peer-synced messages
- Chain hydration path now broadcasts SSE so gate messages arriving via
public chain sync trigger frontend refresh
- Node participation defaults to enabled so fresh installs automatically
join the mesh network (push + pull)
Relay nodes run in store-and-forward mode with no local gate configs,
so gate_manager.can_enter() always returned "Gate does not exist" —
silently rejecting every pushed gate message. This broke cross-node
gate message delivery entirely since no relay ever stored anything.
Relay mode now skips the gate-existence check after signature
verification passes, allowing encrypted gate blobs to flow through.
Repo migration in March 2026 rewrote all commit hashes, leaving old
clones with a docker-compose.yml that builds from source instead of
pulling pre-built images. Added detection warnings to compose.sh,
start.bat, and start.sh so affected users see clear instructions.
Also exposes APP_VERSION in /api/health for easier debugging.
- Add Server-Sent Events endpoint at GET /api/mesh/gate/stream that
broadcasts ALL gate events to connected frontends (privacy: no
per-gate subscriptions, clients filter locally)
- Hook SSE broadcast into all gate event entry points: local append,
peer push receiver, and pull loop
- Reduce push/pull intervals from 30s to 10s for faster relay sync
- Add useGateSSE hook for frontend EventSource integration
- GateView + MeshChat use SSE for instant refresh, polling demoted
to 30s fallback
Latency: same-node instant, cross-node ~10s avg (was ~34s)
Nodes behind NAT could push gate messages to relays but had no way
to pull messages from OTHER nodes back. The push loop only sends
outbound; the public chain sync carries encrypted blobs but peer-
pushed gate events never made it onto the relay's chain.
Adds:
- POST /api/mesh/gate/peer-pull: HMAC-authenticated endpoint that
returns gate events a peer is missing (discovery mode returns all
gate IDs with counts; per-gate mode returns event batches).
- _http_gate_pull_loop: background thread (30s interval) that pulls
new gate events from relay peers into local gate_store.
This closes the loop: push sends YOUR messages out, pull fetches
EVERYONE ELSE's messages back.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The gate_peer_push endpoint was stripping gate_envelope and reply_to
from incoming events, making cross-node message decryption impossible.
Messages would arrive but couldn't be read by the receiving node.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use cipher0's existing MESH_PEER_PUSH_SECRET so nodes connect
to the relay out of the box without configuration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Phase 1 — Transport layer fix:
- Bake in default MESH_PEER_PUSH_SECRET so peer push, real-time
propagation, and pull-sync all work out of the box instead of
silently no-oping on an empty secret.
- Pass secret through docker-compose.yml for container deployments.
Phase 2 — Per-gate content keys:
- Generate a cryptographically random 32-byte secret per gate on
creation (and backfill existing gates on startup).
- Upgrade HKDF envelope encryption to use per-gate secret as IKM
so knowing a gate name alone no longer decrypts messages.
- 3-tier decryption fallback (phase2 key → legacy name-only →
legacy node-local) preserves backward compatibility.
- Expose gate_secret via list_gates API for authorized members.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Derive gate envelope AES key from gate ID via HKDF so all nodes
sharing a gate can decrypt each other's messages (was node-local)
- Preserve gate_envelope/reply_to in chain payload normalization
- Bump Wormhole modal text from 9-10px to 12-13px
- Add aircraft icon zoom interpolation (0.8→2.0 across zoom 5-12)
- Reduce Mesh Chat panel text sizes for tighter layout
The MLS gate encryption system requires libprivacy_core.so — a Rust
shared library that was only compiled locally on the dev machine.
Docker users got "active gate identity is not mapped into the MLS
group" because the library was never built or included in the image.
Add a multi-stage Docker build:
- Stage 1: rust:1.87-slim-bookworm compiles privacy-core to .so
- Stage 2: copies libprivacy_core.so into the Python backend image
- Set PRIVACY_CORE_LIB env var so Python finds the library
Also track the privacy-core Rust source (Cargo.toml, Cargo.lock,
src/lib.rs) in git — they were previously untracked, which is why
the Docker build never had access to them.
Add root .dockerignore to exclude build caches and large directories
from the Docker build context.
On a fresh Docker (or local) install, MESH_RELAY_PEERS was empty and
no bootstrap manifest existed, leaving the Infonet node with zero
peers to sync from — causing perpetual "RETRYING" status.
Set cipher0.shadowbroker.info:8000 as the default relay peer in both
the config defaults and docker-compose.yml so new installations sync
immediately after activating the wormhole.
The Meshtastic MQTT bridge was using client.loop(timeout=1.0) in a
blocking while loop. When the broker dropped the connection (common
after ~30s of idle in Docker), the client silently stopped receiving
messages with no auto-reconnect.
Switch to client.loop_start() which runs the MQTT network loop in a
background thread with built-in automatic reconnection. Also:
- Add on_disconnect callback for visibility into disconnection events
- Set reconnect_delay_set(1, 30) for fast exponential-backoff reconnect
- Lower keepalive from 60s to 30s to stay within Docker network timeouts
Full import audit found these packages used but missing from
pyproject.toml — all silently broken in Docker:
- meshtastic: MQTT protobuf decode (why US/LongFast chat was empty)
- PyNaCl: DM sealed-box encryption
- vaderSentiment: oracle sentiment analysis (unguarded, would crash)
paho-mqtt v2 changed Client constructor and on_connect callback
signatures, breaking the Meshtastic MQTT bridge. Pin to <2.0.0
so the existing v1 code works correctly in Docker.
paho-mqtt was missing from pyproject.toml, causing the Meshtastic MQTT
bridge to silently disable itself in Docker — no live chat messages
could be received. Also improve Infonet node status labels: show
RETRYING when sync fails instead of misleading SYNCING, and WAITING
when node is enabled but no sync has run yet.
Docker/Linux containers have no DPAPI or native keyring, causing all
wormhole persona/gate/identity endpoints to crash with
SecureStorageError. Detect /.dockerenv and auto-allow raw fallback
so mesh features work out of the box in Docker.
In Docker the wormhole subprocess takes 10-15s to start (loading
Plane-Alert DB, env checks, uvicorn startup). The 8s deadline was
expiring before the health probe could succeed, leaving ready=false
permanently even though the subprocess was healthy.
Exit early from _ais_stream_loop() if AIS_API_KEY is empty instead of
endlessly spawning the Node proxy which immediately prints FATAL and
exits. This was flooding docker logs with hundreds of lines per minute.
- require_local_operator now recognizes Docker bridge network IPs
(172.x, 192.168.x, 10.x) as local, fixing "Forbidden — local operator
access only" when frontend container calls wormhole/mesh endpoints
- Bumped all changelog modal text from 8-9px to 11-13px for readability
Changed _validate_admin_startup() from sys.exit(1) to a warning when
ADMIN_KEY is not set. Regular dashboard users don't need admin/mesh
endpoints — the app should start and serve the dashboard without them.
@aaronjmars