mirror of
https://github.com/BigBodyCobain/Shadowbroker.git
synced 2026-05-28 10:01:31 +02:00
c54ea7fd9f6e27e586fe82b767da33bf29f65427
38 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 BigBodyCobain
|
c54ea7fd9f |
Fix #299/#300/#301: gate Sentinel proxy routes with require_local_operator
Reported by @tg12 in three audit issues opened the same day: #299 — POST /api/sentinel/token is an unauthenticated Copernicus OAuth relay for caller-supplied client_id/secret. #300 — POST /api/sentinel/tile is an unauthenticated quota/bandwidth relay for Sentinel Hub Process API tile fetches. #301 — GET /api/sentinel2/search is an unauthenticated Planetary Computer STAC + Esri imagery search relay. All three lived in backend/routers/tools.py decorated only with @limiter.limit(...) — no Depends(require_local_operator). That made the backend a free anonymous relay for any caller's Sentinel / Planetary Computer queries, in the same shape we already closed for #240/#241 (oracle resolve) and #211/#213/#214 (thermal verify, OpenMHZ calls + audio relay). Fix: add dependencies=[Depends(require_local_operator)] to each route. Loopback / Docker-bridge / admin-key callers (the operator dashboard) are unaffected — they still resolve through the same allowlist used by every other operator-only helper in this file. Anonymous remote callers now receive 403 BEFORE any outbound HTTP call to Copernicus or Planetary Computer happens. Tests ----- test_sentinel_routes_auth_gate.py — 8 new tests: * anonymous-remote → 403 on all three routes * NO upstream HTTP call when the gate fires (asserted via MagicMock(side_effect=AssertionError) on requests.post and services.sentinel_search.search_sentinel2_scene). This is the property that makes the gate real — without it, a 403 returned after the upstream call still burns quota. * 127.0.0.1 loopback caller reaches the handler (no false-positive where the gate accidentally blocks the local operator too). * Uses raw ASGITransport(client=(peer_ip, ...)) rather than FastAPI's TestClient because TestClient reports client.host as "testclient" which is not on the loopback allowlist. test_control_surface_auth.py — extended the existing parameterised regression with the three new routes. That regression is the global "no remote control surface ships without auth" guard for the whole codebase; adding these to it means a future refactor that drops the dependency from any of them will fail CI alongside the existing ~30 gated routes. The egress-on-403 property and the parameterised regression together give two independent proofs that the gate fires before the upstream network call, even if FastAPI's internal dependant tree shape changes across versions (an earlier draft of this PR included a static walker of the route table; it was removed because behavioural evidence is strictly stronger and version-independent). |
||
|
Shadowbroker
|
19fb7f0b1e |
Fix #288: viewport-scoped live-data for heavy layers only (#294)
Reported by @tg12 in the external security/correctness audit.
Before this change, /api/live-data/{fast,slow} accepted s/w/n/e query
params but their Query() descriptions explicitly said "(ignored)". The
endpoints shipped the full in-memory world dataset on every poll:
/api/live-data/fast → 16.88 MB
/api/live-data/slow → 10.12 MB
── 27 MB per poll cycle, regardless of zoom
For a node with N operators each polling at the steady 15s/120s cadence,
this is hundreds of MB/minute of outbound traffic that never gets used —
the GPU just culls everything outside the viewport client-side. On a
Tor-bridged or LTE-backed node, that bandwidth bill is the actual cost.
This change makes the existing s/w/n/e params honored — when all four
bounds are supplied, the backend bbox-filters a curated set of heavy,
density-driven, time-sensitive collections to that viewport (with the
existing 20% padding from _bbox_filter):
/fast: commercial_flights, military_flights, private_flights,
private_jets, tracked_flights, ships, cctv, uavs, liveuamap,
gps_jamming, sigint, trains
/slow: gdelt, firms_fires, kiwisdr, scanners, psk_reporter
Static reference layers (satellites, datacenters, military_bases,
power_plants, satnogs, weather, news, stocks, etc.) deliberately STAY
world-scale so panning never reveals an "empty world" of infrastructure.
That preserves the no-hostile-UX feel of the existing dashboard.
Behavior contract:
* Without bbox params (or with a partial bbox), the response is
byte-for-byte identical to the pre-#288 implementation. No
behavior change for any existing caller that hasn't opted in.
* World-scale bbox (lng_span >= 300 or lat_span >= 120) short-circuits
filtering and shares the global ETag — zoomed-out operators all
hit the same 304 cache exactly like before.
* ETag now mixes a 1°-quantized bbox suffix when filtering engages,
so two viewports never poison each other's 304 cache. Sub-degree
pans land in the same ETag bucket (i.e. don't bust the cache on
every mouse drag).
Polling cadence, rate-limit windows, and the 304 short-circuit are all
unchanged. Only the SIZE of the responses changes, and only when the
caller opts in via bounds.
Frontend wiring: useViewportBounds reuses the same coarsened/
expanded bounds it already computes for the AIS /api/viewport POST and
pushes them into a new module-level liveDataViewport store.
useDataPolling reads from that store via appendLiveDataBoundsParams
when building each live-data URL.
Tests cover: no-bbox → world data; bbox → heavy layers filtered;
bbox → reference layers untouched; world-scale bbox → no filter;
partial bbox → treated as no bbox; ETag changes with bbox; sub-degree
pan → same ETag; 304 path works; antimeridian-crossing bbox handled.
Co-authored-by: BigBodyCobain <moatbc@gmail.com>
|
||
|
Shadowbroker
|
35cd4e4c71 |
Fix #287: proxy-aware rate-limit key (#295)
Reported by @tg12 in the external security/correctness audit.
Before this change, backend/limiter.py was:
from slowapi.util import get_remote_address
limiter = Limiter(key_func=get_remote_address)
get_remote_address only ever returns request.client.host — it does
not look at X-Forwarded-For. Behind the bundled Next.js proxy (or any
other reverse proxy), every connected operator's client.host is the
frontend container's bridge IP, so @limiter.limit("120/minute")
collapses into one shared bucket for everybody on the same backend.
One heavy tab can starve every other operator on that node.
This change swaps in shadowbroker_rate_limit_key, which:
* Reads X-Forwarded-For ONLY when the immediate peer matches the
SAME hostname-bound allowlist we use for Docker-bridge local-operator
trust (auth._resolve_trusted_bridge_ips — fix #250). Default is
`frontend,shadowbroker-frontend`, override via
SHADOWBROKER_TRUSTED_FRONTEND_HOSTS.
* Picks the FIRST entry in the XFF chain — that's the operator end,
not the proxy end.
* Falls back to request.client.host for any peer not on the
allowlist. Direct hits, unrelated containers, and unknown hosts
are bucketed exactly like before.
* Falls back to request.client.host when the resolver itself raises
(e.g. DNS down). XFF is never accepted on a peer we can't confirm
is the trusted frontend — there is no way to spoof another
operator's bucket from outside.
No new env vars. No new operator config. Single-operator nodes are
unaffected — same behaviour as before. The 120/minute and 60/minute
windows on the existing endpoints are unchanged; only the KEY they
bucket on changes.
Tests cover:
* Direct loopback → keys on peer (regression check vs.
get_remote_address default).
* Untrusted peer sending XFF → XFF ignored, keys on peer.
* Trusted frontend peer with XFF → keys on first XFF entry.
* First XFF entry picked from a multi-hop chain.
* Trusted peer without XFF → falls back to peer IP.
* Empty/whitespace XFF entries skipped.
* Header lookup is case-insensitive.
* Two operators behind same proxy → different keys (the whole
point of the fix).
* Spoof attempt from internet-facing untrusted IP can't steal the
victim's bucket.
* Resolver raising is treated as untrusted (fail-closed).
* No-client request shape doesn't raise.
Co-authored-by: BigBodyCobain <moatbc@gmail.com>
|
||
|
Shadowbroker
|
76750caa92 |
Round 7a: per-operator outbound attribution + GDELT GCS-direct fix (#292)
== Per-install operator handle for every third-party API call ==
Before this PR, every Shadowbroker install identified itself to
Wikipedia, Wikidata, Nominatim, GDELT, OpenMHz, Broadcastify,
weather.gov, NUFORC, Sentinel/Planetary Computer, TinyGS / CelesTrak,
Shodan, Finnhub, and others with a single project-wide User-Agent
("Shadowbroker/1.0" or "ShadowBroker-OSINT/1.0"). From the upstream's
perspective every install in the world looked like one giant scraper.
If one install misbehaved, the upstream's only recourse was to block
"Shadowbroker" as a whole.
PR #284 inadvertently doubled down on this in the frontend by
introducing a shared `WIKIMEDIA_API_USER_AGENT` constant. This PR
retrofits both backends to per-operator attribution.
New setting: OPERATOR_HANDLE (env var / settings UI / auto-gen)
New helper: network_utils.outbound_user_agent("purpose")
The handle is auto-generated as "operator-XXXXXX" on first call (the
"shadow-" prefix from earlier drafts was deliberately dropped — too
suspicious-looking for abuse-detection systems). Operators can
override via OPERATOR_HANDLE; the value is sanitized to lowercase
alphanumeric+dash+underscore and capped at 48 chars. Persisted to
backend/data/operator_handle.json so it survives container restarts.
Retrofitted call sites (every previously-MONSTER User-Agent):
- services/region_dossier.py (Wikipedia + Wikidata + Nominatim)
- services/geocode.py (Nominatim)
- services/sentinel_search.py (Microsoft Planetary Computer)
- services/feed_ingester.py (operator-curated RSS feeds)
- services/fetchers/earth_observation.py (weather.gov, NUFORC)
- services/fetchers/infrastructure.py
- services/fetchers/aircraft_database.py
- services/fetchers/route_database.py
- services/fetchers/trains.py
- services/fetchers/meshtastic_map.py
- services/shodan_connector.py
- services/unusual_whales_connector.py (Finnhub)
- services/tinygs_fetcher.py (CelesTrak + TinyGS)
- services/sar/sar_products_client.py
- services/geopolitics.py (GDELT)
- services/radio_intercept.py (Broadcastify + OpenMHz)
- routers/cctv.py + main.py (CCTV proxy)
- routers/ai_intel.py
- scripts/convert_power_plants.py (release-time data refresh)
Spoofed browser UAs removed (issues #289 / #290 / #291 — tg12 audit):
- cloudscraper-based Chrome impersonation against api.openmhz.com
-> replaced with honest requests + per-install UA
- Mozilla/5.0 spoofed UA on Broadcastify scrape
-> replaced with honest UA
- Mozilla/5.0 + fake first-party Referer on OpenMHz audio relay
-> replaced with honest UA
- cloudscraper dependency dropped from pyproject.toml + uv.lock
Frontend retrofit:
- new GET /api/settings/operator-handle endpoint (local-operator
gated) returns the install's handle
- frontend/src/lib/wikimediaClient.ts fetches the handle once on
first use, caches it for page lifetime, embeds it in the
Api-User-Agent for every Wikipedia / Wikidata browser-direct call
== GDELT GCS-direct fix ==
GDELT's data.gdeltproject.org is a CNAME to a Google Cloud Storage
bucket. GCS responds with the wildcard *.storage.googleapis.com cert
which legitimately does NOT cover the GDELT custom domain, so Python's
TLS verification correctly refuses the connection. Some networks
happen to route through a path where this works; many (notably Docker
Desktop's outbound NAT on local installs) do not. Verified on the
maintainer's local install: GDELT was unreachable; 1610 geopolitical
events / 48 export files were dropping silently.
Fix: services/geopolitics._gcs_direct_gdelt_url() rewrites any
data.gdeltproject.org URL to its GCS-direct equivalent
(storage.googleapis.com/data.gdeltproject.org/...) where the standard
GCS cert is genuinely valid. api.gdeltproject.org and every other host
are left untouched.
Confirmed live: backend log goes from
GDELT lastupdate failed: 500
to
Downloading 48 GDELT export files...
Downloaded 48/48 GDELT exports
GDELT parsed: 1610 conflict locations from 48 files
== Tests ==
backend/tests/test_per_operator_outbound_attribution.py (12 tests)
backend/tests/test_gdelt_gcs_direct_rewrite.py (6 tests)
backend/tests/test_region_dossier_wikimedia_ua.py (updated to
pin the helper + per-operator handle, not the old constant)
frontend/src/__tests__/utils/wikimediaClient.test.ts (rewritten
to mock /api/settings/operator-handle and assert per-operator UA)
Local: backend 114/114 security+audit+round7a suite green;
frontend 718/718 vitest suite green.
Credit: tg12 (external security audit, issues #289/#290/#291
relating to spoofed UAs); BigBodyCobain (operator-prefix call,
GDELT cloud-vs-local diagnosis).
|
||
|
Shadowbroker
|
c3ef9f4b9e |
Fix #239: CI guard against new duplicate route registrations (#286)
The audit's concern is that FastAPI behavior depends on the order
routes are registered, because backend/main.py and several router
modules register the same (method, path) pairs twice.
Empirical verification (done in this PR's investigation, see
test_router_handler_is_the_one_that_serves) shows:
- main.app.include_router(...) runs at line ~3316.
- All @app.get/post/... decorators in main.py run AFTER that.
- FastAPI matches in registration order -> the router handler always
wins; the main.py copies are dead code at the route-resolution
layer.
So behavior today is deterministic, but drift between the two copies
is a real future risk: someone editing only one copy of a pair
introduces silent inconsistency, exactly as we saw in round 5 with
_WORMHOLE_PUBLIC_SETTINGS_FIELDS (which existed in BOTH main.py and
routers/wormhole.py and had to be tightened in both).
This PR is the lowest-risk fix: a CI guard that captures today's 166
known duplicates as a baseline and fails the build if any NEW
duplicate appears later. Existing duplicates are tolerated. Removed
duplicates are allowed (the baseline is a ceiling, not a floor). No
production code is deleted or moved -- the dedup of the existing 166
duplicates can be staged separately in future PRs without rushing.
Files:
- backend/tests/data/duplicate_routes_baseline.json
Snapshot of every currently-tolerated (METHOD path) duplicate with
the modules that register each copy. Generated from a live import
of main.app via the snippet in the test docstring.
- backend/tests/test_no_new_duplicate_routes.py
Three tests:
1. test_no_new_duplicate_route_registrations -- the actual guard,
fails if (METHOD, path) not in baseline is found duplicated.
2. test_baseline_only_lists_real_duplicates -- warns (does not
fail) if the baseline has entries that no longer correspond to
a real duplicate; informational housekeeping for the next
baseline regeneration.
3. test_router_handler_is_the_one_that_serves -- pins the
empirical claim that for every duplicated path the router
handler is the first-registered one. If someone ever reorders
include_router() to come AFTER @app decorators, this test
fails loudly and points at the most likely cause.
Verified locally:
- 3/3 new tests pass with current main (166 baselined dups).
- Synthetic duplicate injected into main.app at runtime IS caught by
test 1.
- Full security+carrier suite (96 tests) still green.
Credit: tg12 (external security audit).
|
||
|
Shadowbroker
|
5e6bb8511a |
Fix #244/#245/#246: carrier tracker seed/cache/freshness model (#285)
Replace the dated editorial fallback positions baked into the registry with a one-shot seed file + persistent observation cache. The user's runtime cache now reflects what THIS install has actually observed, not what USNI published on March 9, 2026. A year from now, the cache holds a year of observations and the seed is irrelevant. == #244: dated editorial coordinates out of the registry == CARRIER_REGISTRY no longer carries fallback_lat/lng/heading/desc. Those fields are deleted. The registry is now identity + homeport only. New file: backend/data/carrier_seed.json - Read-only, shipped with every release. - Used ONCE on first-ever startup to bootstrap carrier_cache.json. - Each entry stamped with position_confidence="seed" and the actual as-of date (2026-03-09), NOT now(). == #245: approximate confidence for headline-derived positions == _parse_carrier_positions_from_news() now stamps every GDELT-derived entry with position_confidence="approximate" so the UI knows the coordinate is a region-centroid match, not a precise observation. After the freshness window the label rolls over to "stale_approximate" so old-and-imprecise is distinguishable from recent-and-imprecise. The article's actual seendate is used as position_source_at instead of now(), so the "last reported X days ago" badge is honest. == #246: freshness is labelling, not eviction == The cache always preserves the last position the system observed, forever. What changes is the position_confidence label: - within configurable window (default 14d, env-overridable via SHADOWBROKER_CARRIER_FRESHNESS_DAYS) -> "recent" - older -> "stale" - seed-bootstrap entries that were never refreshed -> "seed" - homeport defaults (carrier added post-install) -> "homeport_default" - headline-derived (any age, fresh) -> "approximate" - headline-derived (older than window) -> "stale_approximate" The position itself never reverts to the seed or the registry. The user always sees the last position the system observed. Per the user's explicit guidance: "from there have it be the last position the user has logged the carriers that way a year from now it doesnt revert to where the ships are today". == Other improvements == - CACHE_FILE moved to backend/data/carrier_cache.json so it lives in the volume-mounted dir under Docker compose. Previously it was at /app/carrier_cache.json which got wiped on every container restart (pre-existing bug). - Atomic cache write (temp + os.replace) so a crash mid-write does not leave a truncated cache file. == Public API shape == Every carrier object the API emits now includes: - position_confidence: seed | recent | stale | approximate | stale_approximate | homeport_default - position_source_at: ISO timestamp of when the underlying source was observed (NOT now()) - is_fallback: convenience boolean for the UI; true when the confidence is seed/stale/stale_approximate/ homeport_default Existing fields (estimated, source, source_url, last_osint_update, name, type, lat, lng, country, desc, wiki) are preserved exactly so the current ShipPopup frontend renders unchanged. last_osint_update now reflects position_source_at instead of now(), which is what the existing "last reported MM/DD" badge always meant to show. Tests: backend/tests/test_carrier_tracker_quality.py — 17 tests covering seed bootstrap, subsequent-startup ignoring seed, no-seed/ no-cache homeport fallback, registry no longer has fallback fields, freshness window labelling + env override, "year-old cache entry keeps its position, only the label flips" regression, approximate confidence for headline matches, GDELT seendate ISO parser, public response shape backward compat. Credit: tg12 (external security audit, three P1/P2 issues). |
||
|
Shadowbroker
|
0fee36e8f7 |
Fix #218/#219/#220: identify ShadowBroker on Wikipedia + Wikidata calls (#284)
Wikimedia's User-Agent policy asks API clients to identify themselves with a stable, contactable identifier so their operators can rate-limit or coordinate. Before this change, ShadowBroker was sending: - Backend (region_dossier.py): generic project default UA only; no Api-User-Agent. - Frontend (useRegionDossier.ts, WikiImage.tsx, NewsFeed.tsx): zero identifying header at all; three separate copy-pasted anonymous fetches with their own module-local caches. Three separate components doing the same broken thing meant policy fixes had to happen in three places, with no shared cache or kill switch. Fix (no UX change, zero hostility): == Backend == `backend/services/region_dossier.py` now sets explicit `User-Agent` + `Api-User-Agent` headers on every outbound Wikidata and Wikipedia request via a new `_WIKIMEDIA_REQUEST_HEADERS` constant. The identifier includes a contact path (issues page on the public GitHub repo). == Frontend == New shared helper `frontend/src/lib/wikimediaClient.ts`: - `fetchWikipediaSummary(title)` — single source of truth for Wikipedia REST summary lookups, with one shared LRU cache (in-flight requests deduplicated, 512-entry cap), `Api-User-Agent` on every fetch. - `fetchWikidataSparql(query)` — same shape for Wikidata SPARQL. - `WIKIMEDIA_API_USER_AGENT` — exported constant; one place to update if Wikimedia ever asks us to back off. Refactored three components to use the shared client: - `frontend/src/hooks/useRegionDossier.ts` — fetchLeader() and fetchLocalWikiSummary() now route through the shared helpers. - `frontend/src/components/WikiImage.tsx` — uses fetchWikipediaSummary, proper React state instead of module-mutation + forceUpdate trick. - `frontend/src/components/NewsFeed.tsx` — same shape. UX: byte-for-byte identical. Same thumbnails, same dossier content, same load behavior. The only observable difference is the outgoing request header. Note on #239 (route duplication): an audit-grade inventory shows 166 main.py routes are shadowed by router modules. That cleanup is too large to land safely in this PR; it will be staged as a separate ladder of small PRs grouped by router module. Tests: - `backend/tests/test_region_dossier_wikimedia_ua.py` — 3 tests asserting backend headers are present. - `frontend/src/__tests__/utils/wikimediaClient.test.ts` — 9 tests covering Api-User-Agent presence, shared cache, concurrent deduplication, disambiguation/HTTP-error/network-error fallthroughs, empty-input safety. Local: backend 76/76 security suite green, frontend 716/716 vitest suite green. Credit: tg12 (external security audit). |
||
|
Shadowbroker
|
e125467721 |
Fix #243/#252/#253: stop leaking settings posture to anonymous callers (#283)
Three settings endpoints were disclosing operational posture or operator-curated configuration to any network caller. This change either tightens the redacted-public view (#243) or adds a local-operator auth gate (#252, #253) per the audit recommendations. Zero hostility to legitimate users: in all three cases, the Tauri shell (loopback), the Docker bridge frontend container (#250 + #278), and any caller with an admin key continue to see the full data. Only anonymous LAN/internet callers see the reduced surface. == #243 (Wormhole transport posture, anonymous-mode, profile, node mode) Tightened the public-redaction allowlists in BOTH the main.py and routers/wormhole.py copies: - _WORMHOLE_PUBLIC_SETTINGS_FIELDS: {enabled, transport, anonymous_mode} -> {enabled} - _WORMHOLE_PUBLIC_PROFILE_FIELDS: {profile, wormhole_enabled} -> {wormhole_enabled} `GET /api/settings/node` (both the routers/admin.py and main.py copies) now returns an empty stub for unauthenticated callers and the full node_mode + node_enabled fields only for authenticated callers via _scoped_view_authenticated(request, "node"). == #252 (news feed inventory disclosure) `GET /api/settings/news-feeds` now requires Depends(require_local_operator) in both the canonical routers/admin.py handler and the duplicate main.py handler. Anonymous callers can no longer enumerate operator-curated feed names and URLs. == #253 (Time Machine archival-capture posture disclosure) `GET /api/settings/timemachine` now requires Depends(require_local_operator). Anonymous callers can no longer fingerprint whether a deployment is retaining replayable historical surveillance data. Tests: backend/tests/test_round5_settings_info_disclosure.py (10 tests) - Wormhole settings: anonymous sees only `enabled`; authenticated sees full state. - Privacy profile: anonymous sees only `wormhole_enabled`; authenticated sees `profile` + `transport` + `anonymous_mode`. - Node settings: anonymous sees `{}`; authenticated sees node_mode + node_enabled + persisted state. - news-feeds: anonymous gets 403 (and get_feeds() is NOT called); authenticated gets full inventory. - timemachine: anonymous gets 403; authenticated sees enabled + storage_warning. Local: 73/73 security suite (round 5 + earlier rounds) green. Credit: tg12 (external security audit, P1 + 2x Medium). |
||
|
Shadowbroker
|
2e14e75a0e |
Fix #256: per-peer HMAC secrets defeat cross-peer impersonation (#281)
Before this change, every peer-push HMAC was derived from the single fleet-shared MESH_PEER_PUSH_SECRET. The receiver could prove "this request was signed by someone who knows the fleet secret" but it could NOT prove which peer signed it. Any peer that knew the global secret could compute the expected HMAC for any other peer URL and forge a push pretending to be that peer. Fix: introduce MESH_PEER_SECRETS, an optional comma-separated url=secret map. When a peer URL appears in the map, only the listed per-peer secret is accepted for it -- the global secret is ignored for that specific URL. Peer A no longer knows peer B's secret, so peer A cannot forge a push claiming to be peer B. The new helper resolve_peer_key_for_url() in mesh_crypto.py wraps the lookup and is called from every existing peer-push call site: - backend/auth.py:_verify_peer_push_hmac (receiver) - backend/main.py:_http_peer_push_loop (Infonet event push) - backend/main.py:_http_gate_pull_loop (gate event pull) - backend/main.py:_http_gate_push_loop (gate event push) - backend/services/mesh/mesh_router.py (two transports, push) - backend/services/mesh/mesh_hashchain.py (gate wire ref key) - backend/services/mesh/mesh_wormhole_prekey.py (peer prekey lookup) Zero hostility, by design: - Single-peer installs leave MESH_PEER_SECRETS empty -> resolver falls back to MESH_PEER_PUSH_SECRET -> behavior is byte-for-byte unchanged. - Multi-peer installs that haven't migrated yet behave exactly as before. - Multi-peer installs that DO migrate set MESH_PEER_SECRETS on both ends of each peering and immediately close the impersonation surface for those URLs. Migration is incremental: unlisted peers keep using the global secret. Tests in backend/tests/test_per_peer_secret_resolver.py: - env parsing (default, override, whitespace, malformed entries, cache) - precedence: per-peer beats global - migration window: unlisted peer falls back to global - IMPERSONATION REFUSAL: peer A with global-secret-only cannot forge HMAC for peer B that has a per-peer secret configured - IMPERSONATION REFUSAL: peer A with its OWN per-peer secret cannot forge HMAC for peer B - positive control: legitimate peer B request verifies - zero-behavior-change: single-peer install produces the same key bytes as before the change Credit: tg12 (external security audit, P1/High/High confidence) |
||
|
Shadowbroker
|
084e563412 |
Fix #240/#241: require admin auth on oracle resolve endpoints (#280)
Both POST /api/mesh/oracle/resolve and POST /api/mesh/oracle/resolve-stakes were previously gated only by a rate limit (5/min) and tagged with `mesh_write_exempt(MeshWriteExemption.ADMIN_CONTROL)`. The exemption decorator is metadata only — it tells the mesh signed-write middleware not to require a signature envelope, it does NOT enforce caller authorization. Any network caller could: - /resolve: settle any prediction market to any outcome (corrupts every downstream profile/win-loss count derived from that ledger). - /resolve-stakes: trigger stake settlement for all expired contests at a time of their choosing (race against operator intent). Fix: add `dependencies=[Depends(require_admin)]` to both routes. The existing `mesh_write_exempt` tag stays in place because it accurately describes the route's relationship to the signed-write envelope system; adding `require_admin` is what closes the actual auth hole. Tests in backend/tests/test_oracle_resolve_auth_gate.py: - anonymous caller -> 403, ledger mutator NOT called - wrong admin key -> 403, ledger mutator NOT called - valid admin key -> 200, ledger mutator called - admin key unconfigured + no debug/insecure-admin -> 403 Credit: tg12 (external security audit) |
||
|
Shadowbroker
|
9ef6213284 |
Fix #250: bind Docker bridge local-operator trust to frontend hostname (#278)
Tightens the bridge-trust check so a connection on the Docker bridge is only granted local-operator status when its source IP matches a configured frontend container hostname (default: `frontend` + the shipped `container_name` `shadowbroker-frontend`). Previously, when `SHADOWBROKER_TRUST_DOCKER_BRIDGE_LOCAL_OPERATOR=1` was set, ANY IP in the 172.16.0.0/12 range was granted local-operator privileges — on a shared Docker host that included any unrelated container on the same bridge. Operators with renamed services can list new hostnames via the new `SHADOWBROKER_TRUSTED_FRONTEND_HOSTS` env var (comma-separated). DNS resolution is cached for 30s; if Docker DNS can't resolve any of the configured names we fail closed and refuse the bridge entirely. Single-user installs see no behavior change — the default-named frontend container still resolves and is still trusted. Credit: tg12 (external security audit) |
||
|
Shadowbroker
|
fb11e0881f |
Fix #251: refuse symlink/hardlink members during Tor bundle extraction (#277)
External audit (@tg12) flagged that the Tor Expert Bundle extractor
checked tarinfo.name against path traversal but never inspected
tarinfo.linkname for symlink or hardlink members. Python 3.11's
tarfile.extractall() honors symlinks, so a malicious archive could
ship a member like::
name = "innocent.txt" (passes the path-traversal check)
type = SYMTYPE
linkname = "C:\Windows\System32\config\system"
After extraction, subsequent reads of innocent.txt dereference to that
arbitrary filesystem location; subsequent writes corrupt it. On
Windows (where Tor Expert Bundle extraction actually runs), this is
a host-compromise path of essentially the same severity as the
supply-chain RCE in #231 — gated only by the integrity check we just
hardened in PR #261/#265.
Python 3.12+ added tarfile.extract / extractall filter='data' as a
built-in mitigation; we're on Python 3.11 in production, so we
implement the same idea manually.
Fix in backend/services/tor_hidden_service.py:
Extract the existing path-traversal-only check into a new
_extract_tor_bundle_safely() helper that:
1. Refuses any member with member.issym() or member.islnk() True.
Tor bundles never legitimately contain symlinks or hardlinks
so this is non-disruptive. Logs the linkname so an operator
can see what the malicious archive was trying to alias.
2. Refuses any member that isn't isfile() or isdir() — no FIFOs,
no character or block devices, no contiguous-file-type entries.
None of those belong in a Tor Expert Bundle and accepting them
is a class of bug we don't need to debug later.
3. Preserves the original path-traversal guard (member.name must
resolve under install_dir).
4. Catches tarfile.TarError so a corrupt archive returns False
gracefully instead of bubbling out an exception.
Tests: backend/tests/test_tor_bundle_symlink_filter.py (8 tests)
- Clean archive with only regular files extracts successfully
- Symlink member is rejected (the core regression)
- Hardlink member is rejected
- Symlink with relative target inside install_dir is still rejected
(we don't allow symlinks at all, not just absolute-target ones)
- FIFO/device-style member is rejected
- Path-traversal guard still works under the new shape
- Malformed/non-tar file is rejected gracefully (no crash)
- Failure on one member rejects the whole bundle (no half-extract)
Validation:
pytest backend/tests/test_tor_bundle_symlink_filter.py
backend/tests/test_tor_bundle_verification.py
-> 14 passed
UX impact: zero for legitimate Tor releases. Operators installing
a real Tor Expert Bundle continue to see "Tor installed at:" exactly
as before. Only malicious archives are refused, with a clear log
message identifying the rejected linkname.
Credit: @tg12 — the original report was specific enough that the
fix design was immediate.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
Shadowbroker
|
7f96151e56 |
Fix #231: multi-source SHA-256 verification for the self-updater (#265)
External audit (@tg12, May 18) found that backend/services/updater.py silently skipped all SHA-256 integrity verification whenever the MESH_UPDATE_SHA256 env var was unset — which is the default. Nothing in any install doc tells operators to set it, so practically every deployment was running the auto-updater with zero integrity check. That made GitHub release pipeline compromise a single-step path to arbitrary code execution on every node that auto-updates. Investigation surfaced a deeper bug too: the updater downloads zipball_url (GitHub's auto-generated source archive) but the maintainer's release process publishes SHA256SUMS.txt for a separate named asset (ShadowBroker_v*.zip). So even if MESH_UPDATE_SHA256 WERE set, operators had no published digest to compare against — the file they were downloading wasn't the file the maintainer had signed. This PR fixes both issues with the same multi-source verification chain we shipped for the Tor bundle in PR #261: backend/services/updater.py _download_release() now prefers a maintainer-signed release asset matching ShadowBroker_v*.zip over zipball_url. Captures the SHA256SUMS.txt asset URL when present. _validate_zip_hash() rewritten as a four-source chain: 1. MESH_UPDATE_SHA256 env var (operator override, preserved) 2. SHA256SUMS.txt asset published with the release (primary — the maintainer's release process already publishes this) 3. Baked-in backend/data/release_digests.json (second line of defense for releases that lack the SHA256SUMS asset, or when the asset can't be fetched at update time) 4. HTTPS-only fallback with a loud warning (preserves the auto- update flow during transient outages) Mismatch from any source that DID respond is fatal — the update is refused and the existing install keeps running. Only the "no source reachable at all" case falls back to HTTPS-only. _fetch_sha256sums() new — fetches and parses a standard SHA256SUMS.txt asset. Handles both "<digest> <name>" and binary- marker "<digest> *<name>" formats. Tolerant to comments, blank lines, and malformed entries. backend/data/release_digests.json (new) Baked-in digest list keyed by release tag. Seeded with the v0.9.79 entries copied from the published SHA256SUMS.txt: ShadowBroker_v0.9.79.zip = f6877c1d6661... ShadowBroker_0.9.79_x64-setup.exe = f7b676ada45c... ShadowBroker_0.9.79_x64_en-US.msi = e0713c3cdda1... Whitelisted in .gitignore alongside the other static reference data files (kiwisdr_directory.json, tor_bundle_digests.json, aisstream_spki_pins.json). backend/tests/test_update_integrity_chain.py (new, 16 tests) - Each source matches → success, identifies which source verified - Each source mismatches → RuntimeError "mismatch" - No source reachable → https-only fallback with loud warning - Env override beats all other sources (preserved precedence) - SHA256SUMS.txt parser handles standard, binary-marker, comments, and network-failure cases Validation: pytest backend/tests/test_update_integrity_chain.py → 16 passed pytest (all 15 security test files together) → 105 passed UX impact: zero. Normal auto-update flow is unchanged for legitimate releases (path 2 catches everything because the release publishes SHA256SUMS.txt). Transient network failures during update gracefully fall through to path 3 then path 4 — no operator intervention needed. The only user-visible behavior change is in the compromised-release case, where the update is now refused instead of silently applied. Credit: @tg12 for the original bug report and the specific call-out that MESH_UPDATE_SHA256 was unreachable by default operators. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Shadowbroker
|
729ea78cb2 |
Fix #258: AIS proxy SPKI pinning fallback for expired upstream cert (#262)
External report from @jmleclercq: AISStream's Let's Encrypt cert
expired on 2026-05-20 (verified — their renewal pipeline failed), so
the AIS WebSocket connection dies with CERT_HAS_EXPIRED and the
maritime layer empties out. The reporter worked around it locally by
passing { rejectUnauthorized: false } to the WebSocket constructor and
asked whether we should add an env var for that.
That fix is the wrong fix. Disabling TLS validation entirely lets any
network attacker MITM the WebSocket and inject fake ship positions —
same class as the GDELT plaintext-HTTP MITM we just closed in #199.
Adding an env var for it would be an attractive nuisance: operators
set it once during a bad cert week and then forget, leaving themselves
open to MITM forever.
Right fix: SPKI pinning, same pattern as the Tor bundle digest pinning
in #201. The insight is that Let's Encrypt renewals keep the SAME
public key by default, so the SPKI hash survives normal cert rotation.
We can relax the date check while keeping the identity check.
Mechanics:
backend/data/aisstream_spki_pins.json (new)
Pinned SHA-256 hashes of the DER-encoded SPKI bytes for
stream.aisstream.io. Captured 2026-05-20 from the live cert.
Format is base64(sha256(pubkey_der)), matching the canonical
openssl pipeline. Whitelisted in .gitignore alongside the other
static reference data files (KiwiSDR directory, Tor bundle
digests).
backend/ais_proxy.js
Path A (99.9% of the time): normal TLS validation. Untouched.
Path B (on CERT_HAS_EXPIRED only): re-handshake with
rejectUnauthorized=false JUST to read the leaf cert, compute its
SPKI hash, compare against the pinned list. If match → upstream
is still the genuine AISStream → re-open the WebSocket with
rejectUnauthorized=false and log DEGRADED MODE. If no match →
refuse the connection, log loudly: this would be a real MITM.
Pin file is looked up in three locations so the same code works
in the Docker backend, the Tauri desktop runtime, and any
operator-relocated layout (SHADOWBROKER_AIS_PINS env var).
Embedded fallback list inside the JS so portable installs that
haven't shipped the JSON still work.
backend/services/ais_stream.py
Captures the proxy's status markers from stdout
({"__ais_proxy_status": {"degraded_tls": true}}) into a module-
level snapshot. Exposes ais_proxy_status() for the health
endpoint. Doesn't touch the data plane — degraded mode keeps
receiving vessel data, just with weaker MITM protection.
backend/routers/health.py + backend/services/schemas.py
/api/health now includes an ais_proxy block with degraded_tls.
Top-level status escalates ok -> degraded when AIS is in
degraded TLS mode (but won't downgrade a worse SLO status).
Operators get a visible signal that they're in degraded mode
without needing to grep logs.
Tests: backend/tests/test_ais_spki_pinning.py (7 tests)
- Pin file structure validation (JSON, host entry, base64 SHA-256)
- ais_proxy_status() snapshot semantics (starts empty, defensive copy)
- /api/health surfaces ais_proxy.degraded_tls when set
- /api/health returns empty ais_proxy when proxy hasn't reported
Node.js syntax check passes (node --check) on both backend/ais_proxy.js
and the Tauri runtime mirror.
When AISStream renews their cert (likely within hours-to-days), the
normal-TLS path succeeds on next reconnect and degraded_tls clears
automatically. No operator action needed. If they instead rotate their
server key, the SPKI check will fail and we'll need to add the new
hash to backend/data/aisstream_spki_pins.json before removing the old
one.
Credit: @jmleclercq for the clear report and the careful workaround
verification (Node version, ws version, manual probe).
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
Shadowbroker
|
e36d1fc79c |
[security] Close tg12 audit issues #201–#214 seamlessly (#261)
External security audit by @tg12 (May 17, 2026) filed issues #201–#214 in addition to the #189–#200 batch already closed by PRs #227/#232/#260. This PR closes all eight that are real security bugs (the other six in the 201–214 range are either design discussions or upstream-abuse/TOS concerns we're keeping intentional, see issue triage notes on each). The user-facing principle for this PR: fix the security gap WITHOUT introducing a single hostile error or behavior change for legitimate users. Every fix follows the same template — fail forward, not loud. When the secure path is harder than the insecure one, build a fallback chain that ends in graceful degradation, not in a scary modal or 422 response. #205 — OpenMHZ audio redirect SSRF (services/radio_intercept.py) Replaced requests.get(..., allow_redirects=True) with a manual redirect loop that re-validates each hop's host against _OPENMHZ_AUDIO_HOSTS. Same-host redirects (CDN edge selection) still work, so legitimate audio playback is unaffected. Cross-host redirects to disallowed hosts return a generic 502 which the browser audio element handles gracefully. Cap at 5 hops. #207 — infonet/status verify_signatures DoS (routers/mesh_public.py) Silently downgrade verify_signatures=true to False for unauthenticated callers. No error surfaced — the response shape is identical, just without the O(n_events) signature verification. Authenticated callers (scoped mesh.audit) still get the full path. The frontend never passes this param so legitimate UI is unaffected. #211 — thermal/verify expensive analysis (routers/sigint.py) Added Depends(require_local_operator). Frontend has no direct callers (verified by grep); Tauri/AI agents use scoped tokens that pass the auth check. Anonymous abusers blocked silently — the legitimate UI keeps working through the Next.js admin-key proxy. #213, #214 — OpenMHZ calls/audio upstream abuse (routers/radio.py) Added Depends(require_local_operator) to both. Browser users hit these through the Next.js proxy at src/app/api/[...path]/route.ts which injects X-Admin-Key, so the auth check passes transparently. Direct attackers can no longer rotate sys_names to hammer api.openmhz.com or relay arbitrary audio streams through the backend's bandwidth. #202 — overflights unbounded hours (routers/data.py) Silently clamp `hours` to OVERFLIGHTS_MAX_HOURS (default 72, configurable). NO 422 — clients asking for an absurd window get a shorter window back with `requested_hours` and `effective_hours` hint fields. Postel's law: liberal in what we accept, conservative in what we compute. #203 — Meshtastic callsign UA leak (services/fetchers/meshtastic_map.py) Added MESHTASTIC_SEND_CALLSIGN_HEADER opt-out env var. Default is TRUE — preserves existing operator behavior (callsign sent so meshtastic.org can rate-limit per-install). Privacy-conscious operators set it to false to suppress. #206 — KiwiSDR upstream is HTTP-only (services/kiwisdr_fetcher.py) Upstream rx.linkfanel.net doesn't speak HTTPS (verified — Apache 2.4.10 only on port 80). We can't fix the transport. Instead added three layers: 1. Content validation on fetched data — reject responses with <50 receivers or >5% malformed entries (likely MITM injection). 2. Existing disk cache fallback (already present). 3. NEW: bundled static directory at backend/data/kiwisdr_directory.json shipping 798 known-good receivers. Used as last resort so the KiwiSDR map layer always renders something useful. #208 — Merkle proof DoS via /api/mesh/infonet/sync (services/mesh/mesh_hashchain.py) The endpoint is part of the cross-node federation protocol — peers legitimately call it without local-operator auth, so we can't add Depends(). Instead made the underlying operation O(1) per proof via a cached Merkle level structure on the Infonet instance: - _merkle_levels_cache + _merkle_levels_for_event_count on each Infonet instance - _invalidate_merkle_cache() called from every chain mutation point (append, ingest_events, apply_fork, cleanup_expired) - _get_merkle_levels() does the lazy recompute on first read after invalidation, then serves from cache thereafter Effect: anonymous attackers hammering the proofs endpoint hit a cached structure; the rebuild happens at most once per real chain advance. Federation untouched. #201 — Tor bundle SHA-256 bypass (services/tor_hidden_service.py) Docker users were already covered — backend/Dockerfile installs Tor via apt-get at build time (signed by Debian's package system). No runtime download needed for the 80%-of-users case. For Tauri desktop, replaced the single .sha256sum check with a multi-source verification chain implemented in _verify_tor_bundle(): 1. Try upstream .sha256sum (current behavior — fast path) 2. Try baked-in digest list at backend/data/tor_bundle_digests.json (pinned per-version, maintainer-updated) 3. If neither source is REACHABLE: HTTPS-only fallback with a loud warning (avoids breaking first-run onboarding while the maintainer hasn't yet pinned a new Tor release) A mismatch from a source that DID respond is always fatal — only the "no source reachable" case falls back to HTTPS-only. This is the "have cake and eat it" pattern: real users see no new failure modes during torproject.org outages, but MITM/compromise attacks still fail because the downloaded digest can't match what BOTH the upstream and the baked-in list report. Currently the digest file ships with placeholder values for the current Tor URLs (those URLs are already stale on torproject.org too). A follow-up commit can populate real digests when a stable Tor release is selected; until then the HTTPS-only warning fires and onboarding still works. Tests (82 total, all passing): test_openmhz_redirect_ssrf.py (5 tests) — #205 test_infonet_status_verify_gate.py (2 tests) — #207 test_overflights_clamp.py (5 tests) — #202 test_meshtastic_callsign_optout.py (3 tests) — #203 test_kiwisdr_fallback.py (6 tests) — #206 test_merkle_cache.py (6 tests) — #208 test_tor_bundle_verification.py (6 tests) — #201 test_control_surface_auth.py (extended) — #211, #213, #214 + all previous security tests (CCTV redirect, GDELT https, sentinel cache, crowdthreat opt-in, third-party fetcher gates, control surface auth) continue to pass. Pre-existing test infrastructure issue with SHARED_EXECUTOR teardown in the broader sweep exists on main too (verified) — not introduced by this PR. Credit: @tg12 reported every one of these with accurate line citations and the recommended fixes that informed this implementation. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Shadowbroker
|
d00c63abed |
[security] Close tg12 audit gaps #192, #198, #199, #200 (#260)
External security audit by @tg12 (May 17, 2026) filed 11 issues against the backend. PR #227 (May 18, AI-generated) closed seven of them by adding require_local_operator to control-plane endpoints. Four remained live; this PR closes the rest. #192 — CCTV proxy followed redirects without re-validating host Issue: /api/cctv/media validated only the caller-supplied URL host before passing it to requests.get(..., allow_redirects=True). A 302 to http://127.0.0.1 or any internal/disallowed host was silently followed, turning the proxy into an open-redirect-to-SSRF chain. Fix in routers/cctv.py: replace the single allow_redirects=True call with a manual follow loop. Each hop's Location is parsed, the host is rerun through _cctv_host_allowed(), and non-HTTP schemes (file://, ftp://, etc.) are rejected. Cap chain length at 5 hops. Test: backend/tests/test_cctv_redirect_ssrf.py covers - redirect to disallowed host -> 502 - redirect to localhost -> 502 - redirect to another allowed host -> 200 - redirect chain length cap - non-HTTP scheme rejected #198 — Gate introspection GETs were unauthenticated Issue: /api/wormhole/gate/{gate_id}/{identity,personas,key} were callable with no auth dependency. Any caller that could reach the backend could dump the operator's active persona, persona inventory, and key status for any gate_id they knew. The wiki's privacy threat model explicitly markets gate personas as rotating, unlinkable pseudonyms — this leak defeated that property. Fix in routers/wormhole.py: add dependencies=[Depends(require_local_operator)] to all three routes. Test: backend/tests/test_control_surface_auth.py extended with three new parameterized cases (lines 75-77). #199 — GDELT military incident ingestion used plaintext HTTP Issue: backend/services/geopolitics.py fetched http://data.gdeltproject.org/gdeltv2/lastupdate.txt and ~48 export archive URLs over plaintext HTTP. Passive observers could identify Shadowbroker nodes from the fetch pattern. Active MITM could inject doctored military incident records into the global map. Fix in services/geopolitics.py: rewrite the lastupdate.txt fetch and the export download URL constructor to use https://. GDELT's data.gdeltproject.org serves the same content over HTTPS. Test: backend/tests/test_gdelt_https.py asserts no plaintext HTTP URLs to data.gdeltproject.org remain in code (comments excluded) and that the HTTPS URLs we expect are present. #200 — Sentinel token cache lookup used client_id only Issue: routers/tools.py kept a process-global cache of Copernicus bearer tokens. The lookup compared _sh_token_cache["client_id"] == client_id. A caller who knew a valid client_id but supplied any wrong client_secret hit the cache and reused the legitimate caller's bearer token — burning their quota and accessing imagery on their account. Fix in routers/tools.py: replace the client_id field with credential_fp, an HMAC-SHA256 over (client_id, client_secret) under a per-process random key (_SH_TOKEN_CACHE_HMAC_KEY = os.urandom(32), regenerated at startup). A caller who doesn't know the secret cannot compute a matching fingerprint, so they miss the cache and hit the real Copernicus token endpoint — which will reject their wrong secret with a 401. Test: backend/tests/test_sentinel_token_cache.py covers - same client_id + different secrets => different fingerprints - same credentials => same fingerprint (cache still works) - different client_ids + same secret => different fingerprints - cache no longer stores raw client_id (catches regression) - attacker with wrong secret cannot reuse victim's token Validation pytest backend/tests/test_control_surface_auth.py backend/tests/test_cctv_redirect_ssrf.py backend/tests/test_gdelt_https.py backend/tests/test_sentinel_token_cache.py -> 37 passed Credit: @tg12 reported all four of these in their May 17 audit with correct line-number citations and accurate remediation recommendations. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
BigBodyCobain
|
71a9d9e144 |
[security] Close post-#227 control-surface and fetcher gaps
PR #227 hardened most Wormhole/Infonet control surfaces behind require_local_operator and made the CrowdThreat fetcher opt-in. An audit of the codebase against that PR's stated goals turned up four classes of gap that the original change missed: 1. Two operator-only endpoints were left unprotected: - POST /api/wormhole/join: calls bootstrap_wormhole_identity() and flips the node into Tor mode, exactly the surface #227 hardened on /api/wormhole/identity/bootstrap. - POST /api/sigint/transmit: relays APRS-IS packets over radio using operator-supplied credentials. Anything that reached the API could transmit on the operator's authority. Both now require_local_operator. test_control_surface_auth.py extended with regression coverage for both. 2. Five third-party fetchers were still default-on, phoning home to politically/commercially sensitive upstreams on every poll cycle: - fimi.py -> euvsdisinfo.eu -> FIMI_ENABLED - prediction_markets -> Polymarket + Kalshi -> PREDICTION_MARKETS_ENABLED - financial.py -> Finnhub / yfinance -> FINANCIAL_ENABLED or FINNHUB_API_KEY - nuforc_enrichment -> huggingface.co -> NUFORC_ENABLED - news.py -> configured RSS feeds -> NEWS_ENABLED (default on, kill switch) Same CrowdThreat-style pattern: explicit env-var opt-in, empty the data slot and mark_fresh when disabled. New regression test file test_third_party_fetchers_opt_in.py asserts each fetcher's network entry point is not called when its gate is off. 3. The outbound User-Agent leaked both the operator's personal email and a fork-specific GitHub URL on every fetcher request. Consolidated to a single DEFAULT_USER_AGENT in network_utils.py, project-generic by default (no contact info), overridable via SHADOWBROKER_USER_AGENT for operators who want to identify themselves (e.g. for Nominatim or weather.gov usage-policy compliance). Six call sites updated; the Nominatim-specific override is preserved. 4. The same generic UA now also flows through the peer prekey lookup in mesh_wormhole_prekey.py, so DM first-contact requests no longer identify the caller as a Shadowbroker fork to the peer being queried. .env.example updated to document all new opt-in env vars. Tests: backend/tests/test_control_surface_auth.py (extended), backend/tests/test_crowdthreat_opt_in.py (unchanged, still passes), backend/tests/test_third_party_fetchers_opt_in.py (new, 7 tests). All 31 tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
BigBodyCobain
|
11ea345518 | Harden infonet control surfaces | ||
|
BigBodyCobain
|
25a98a9869 |
Harden Infonet DM address flow and seed sync
Allow local-operator DM invite import without requiring a full admin session. Prioritize bundled/bootstrap seed peers and shorten stale seed cooldowns for faster Infonet recovery. Replace raw DM invite dumps with copyable signed-address controls, contact request handling, and safer sealed-send behavior while the private delivery route connects. |
||
|
BigBodyCobain
|
b86a258535 |
Release v0.9.79 runtime and messaging update
Ship the v0.9.79 runtime refresh with transport lane isolation, Infonet secure-message address management, MeshChat MQTT controls, selected asset trail behavior, telemetry panel refinements, onboarding updates, and desktop/package metadata alignment. Also ignore local graphify work products so analysis folders do not leak into future commits. |
||
|
BigBodyCobain
|
b8ac0fb9e7 |
Harden v0.9.75 wormhole node sync and telemetry panels
Add Tor/onion runtime wiring and faster Infonet node status refresh. Keep node bootstrap state clearer across Docker and local runtimes. Use selected aircraft trail history for cumulative tracked-aircraft emissions. |
||
|
BigBodyCobain
|
6ffd54931c |
Release v0.9.75 runtime and onboarding update
Ship the 0.9.75 source update with improved startup/runtime hardening, operator API key onboarding, Meshtastic MQTT controls, Infonet/MeshChat separation, desktop package versioning, and aircraft telemetry refinements. Also updates focused backend/frontend tests for node settings, Meshtastic MQTT settings, and desktop runtime behavior. |
||
|
BigBodyCobain
|
63043b32b5 |
Stabilize Docker startup and runtime proxy
Reduce cold-start stalls by raising the default backend memory limit, bounding heavy feed concurrency, preserving non-empty startup caches, and refreshing working news feeds. Fix the Next API proxy for Docker control-plane writes by stripping unsupported hop/body headers and forwarding small request bodies safely. Keep the dashboard dynamic so production users do not get stuck on a cached startup shell. |
||
|
BigBodyCobain
|
0fc09c9011 | Fix Docker Infonet and Wormhole startup | ||
|
BigBodyCobain
|
707ca29220 |
Add in-app local API key setup
Let fresh Docker and local installs enter OpenSky, AIS, and other provider keys directly in onboarding or Settings without manually creating .env files. Persist keys server-side in the backend data store, keep them write-only from the browser, reload runtime settings, and retain local-operator access controls. |
||
|
BigBodyCobain
|
eb0288ee4e |
Fix Docker local controls and setup guidance
Allow the bundled Docker frontend proxy to reach local-operator endpoints through the private compose bridge without trusting LAN clients. This restores Time Machine, MeshChat key creation, AI pins/layers, and related local controls in Docker installs. Refresh first-run guidance so Docker users know to configure OpenSky and AIS keys through .env. |
||
|
BigBodyCobain
|
e1060193d0 |
Improve v0.9.7 startup and runtime reliability
Prioritize cached first-paint data, defer heavyweight feed synthesis, make MeshChat activation explicit, improve CCTV media handling, and tighten desktop runtime packaging filters. |
||
|
BigBodyCobain
|
28b3bd5ebf | release: prepare v0.9.7 | ||
|
anoracleofra-code
|
668ce16dc7 |
v0.9.6: InfoNet hashchain, Wormhole gate encryption, mesh reputation, 16 community contributors
Gate messages now propagate via the Infonet hashchain as encrypted blobs — every node syncs them through normal chain sync while only Gate members with MLS keys can decrypt. Added mesh reputation system, peer push workers, voluntary Wormhole opt-in for node participation, fork recovery, killwormhole scripts, obfuscated terminology, and hardened the self-updater to protect encryption keys and chain state during updates. New features: Shodan search, train tracking, Sentinel Hub imagery, 8 new intelligence layers, CCTV expansion to 11,000+ cameras across 6 countries, Mesh Terminal CLI, prediction markets, desktop-shell scaffold, and comprehensive mesh test suite (215 frontend + backend tests passing). Community contributors: @wa1id, @AlborzNazari, @adust09, @Xpirix, @imqdcr, @csysp, @suranyami, @chr0n1x, @johan-martensson, @singularfailure, @smithbh, @OrfeoTerkuci, @deuza, @tm-const, @Elhard1, @ttulttul |
||
|
Wa1iD
|
231f0afc4e | fix: restore CCTV layer ingestion and map rendering | ||
|
adust09
|
44147da205 |
fix: resolve merge conflicts between JSDF bases and East Asia adversary bases
Merge both feature sets: keep JSDF bases (gsdf/msdf/asdf branches) from PR #77 and East Asia adversary bases (missile/nuclear branches) from main. Union all branch types in tests and MaplibreViewer labels. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
adust09
|
27506bbaa9 |
test: add JSDF bases tests (RED phase)
- Add gsdf/msdf/asdf to known_branches in test_branch_values_are_known - Add test_includes_jsdf_bases for Yonaguni, Naha, Kure - Add test_colocated_bases_have_separate_entries for Misawa - Add buildMilitaryBasesGeoJSON tests with ASDF branch validation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
adust09
|
910d1fd633 |
feat: enhance East Asia coverage with adversary bases, news sources, ICAO ranges, and PLAN vessel DB
- Add 68 military bases (PLA, Russia, DPRK, ROC, Philippines, Australia) with data-driven color coding (red/blue/green) on the map - Add 6 news RSS feeds (Yonhap, Nikkei Asia, Taipei Times, Asia Times, Defense News, Japan Times) and 15 geocoding keywords for islands, straits, and disputed areas - Extend ICAO country ranges for Russia, Australia, Philippines, Singapore, DPRK and add Russian aircraft classification (fighters, bombers, cargo, recon) - Create PLAN/CCG vessel enrichment module (90+ ships) following yacht_alert pattern for automatic MMSI-based identification - Update frontend types and popup styling for adversary/allied/ROC color distinction Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
adust09
|
4b9765791f |
feat: enrich military aircraft with ICAO country/force and East Asia model classification
Infer country and military force (PLA, JSDF, ROK, ROC) from ICAO hex address blocks when the flag field is Unknown. Extract and extend aircraft model classification to cover East Asian fighters, cargo, recon, and tanker types with hyphen-normalized matching. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
adust09
|
05de14af9d |
feat: add military bases map layer for Western Pacific
Add 18 US military bases (Japan, Guam, South Korea, Hawaii, Diego Garcia) as a toggleable map layer. Follows the existing data center layer pattern: static JSON → backend fetcher → slow-tier API → frontend GeoJSON layer. Includes red circle markers with labels, click popups showing operator and branch info, and a toggle in the left panel. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
adust09
|
130287bb49 |
feat: add East Asia news sources and improve geocoding for Taiwan contingency
Add 5 East Asia-focused RSS feeds (FocusTaiwan, Kyodo, SCMP, The Diplomat, Stars and Stripes) and 22 geographic keywords (Taiwan Strait, South/East China Sea, Okinawa, Guam, military bases, etc.) to improve coverage of Taiwan contingency scenarios. Refactor keyword matching into a pure _resolve_coords() function with longest-match-first sorting so specific locations like "Taiwan Strait" are not absorbed by generic "Taiwan". Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
anoracleofra-code
|
90c2e90e2c |
v0.9.5: The Voltron Update — modular architecture, stable IDs, parallelized boot
- Parallelized startup (60s → 15s) via ThreadPoolExecutor - Adaptive polling engine with ETag caching (no more bbox interrupts) - useCallback optimization for interpolation functions - Sliding LAYERS/INTEL edge panels replace bulky Record Panel - Modular fetcher architecture (flights, geo, infrastructure, financial, earth_observation) - Stable entity IDs for GDELT & News popups (PR #63, credit @csysp) - Admin auth (X-Admin-Key), rate limiting (slowapi), auto-updater - Docker Swarm secrets support, env_check.py validation - 85+ vitest tests, CI pipeline, geoJSON builder extraction - Server-side viewport bbox filtering reduces payloads 80%+ Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Former-commit-id: f2883150b5bc78ebc139d89cc966a76f7d7c0408 |
||
|
anoracleofra-code
|
fc9eff865e |
v0.9.0: in-app auto-updater, ship toggle split, stable entity IDs, performance fixes
New features: - In-app auto-updater with confirmation dialog, manual download fallback, restart polling, and protected file safety net - Ship layers split into 4 independent toggles (Military/Carriers, Cargo/Tankers, Civilian, Cruise/Passenger) with per-category counts - Stable entity IDs using MMSI/callsign instead of volatile array indices - Dismissible threat alert bubbles (session-scoped, survives data refresh) Performance: - GDELT title fetching is now non-blocking (background enrichment) - Removed duplicate startup fetch jobs - Docker healthcheck start_period 15s → 90s Bug fixes: - Removed fake intelligence assessment generator (OSINT-only policy) - Fixed carrier tracker GDELT 429/TypeError crash - Fixed ETag collision (full payload hash) - Added concurrent /api/refresh guard Contributors: @imqdcr (ship split + stable IDs), @csysp (dismissible alerts, PR #48) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Former-commit-id: a2c4c67da54345393f70a9b33b52e7e4fd6c049f |