Compare commits

...

47 Commits

Author SHA1 Message Date
BigBodyCobain 85636ce95c Stabilize secure mail warmup test 2026-05-06 22:54:11 -06:00
BigBodyCobain 5ee4f8ecd7 Stabilize Infonet private sync and selected telemetry 2026-05-06 22:10:04 -06:00
BigBodyCobain b8ac0fb9e7 Harden v0.9.75 wormhole node sync and telemetry panels
Add Tor/onion runtime wiring and faster Infonet node status refresh.

Keep node bootstrap state clearer across Docker and local runtimes.

Use selected aircraft trail history for cumulative tracked-aircraft emissions.
2026-05-06 14:04:16 -06:00
BigBodyCobain 8926e08009 Fix cached satellite propagation 2026-05-06 02:25:10 -06:00
BigBodyCobain 585a08bbac Fix MeshChat decomposition release gate 2026-05-06 01:46:26 -06:00
BigBodyCobain 6ffd54931c Release v0.9.75 runtime and onboarding update
Ship the 0.9.75 source update with improved startup/runtime hardening, operator API key onboarding, Meshtastic MQTT controls, Infonet/MeshChat separation, desktop package versioning, and aircraft telemetry refinements.

Also updates focused backend/frontend tests for node settings, Meshtastic MQTT settings, and desktop runtime behavior.
2026-05-06 01:15:54 -06:00
BigBodyCobain a017ba86d6 Fix desktop release packaging without signing keys 2026-05-04 21:54:29 -06:00
BigBodyCobain 9427935c7f Align CSP tests with hydration-safe policy 2026-05-04 13:04:31 -06:00
BigBodyCobain 63043b32b5 Stabilize Docker startup and runtime proxy
Reduce cold-start stalls by raising the default backend memory limit, bounding heavy feed concurrency, preserving non-empty startup caches, and refreshing working news feeds. Fix the Next API proxy for Docker control-plane writes by stripping unsupported hop/body headers and forwarding small request bodies safely. Keep the dashboard dynamic so production users do not get stuck on a cached startup shell.
2026-05-04 12:37:23 -06:00
BigBodyCobain 1e34fa53b2 Make Docker backend port configurable 2026-05-03 21:13:31 -06:00
BigBodyCobain d69602be9e Align CSP test with production hydration policy 2026-05-03 14:06:39 -06:00
BigBodyCobain ce9ba39cd2 Fix production CSP hydration 2026-05-03 13:59:07 -06:00
BigBodyCobain 3eafb622ed Clarify Podman compose setup 2026-05-03 08:44:56 -06:00
Shadowbroker eb5564ca0e Update README.md 2026-05-03 02:59:03 -06:00
BigBodyCobain 20d2ccc52c Fix desktop static export build 2026-05-02 23:18:57 -06:00
BigBodyCobain 0fc09c9011 Fix Docker Infonet and Wormhole startup 2026-05-02 21:53:35 -06:00
BigBodyCobain 707ca29220 Add in-app local API key setup
Let fresh Docker and local installs enter OpenSky, AIS, and other provider keys directly in onboarding or Settings without manually creating .env files. Persist keys server-side in the backend data store, keep them write-only from the browser, reload runtime settings, and retain local-operator access controls.
2026-05-02 21:16:32 -06:00
BigBodyCobain eb0288ee4e Fix Docker local controls and setup guidance
Allow the bundled Docker frontend proxy to reach local-operator endpoints through the private compose bridge without trusting LAN clients. This restores Time Machine, MeshChat key creation, AI pins/layers, and related local controls in Docker installs. Refresh first-run guidance so Docker users know to configure OpenSky and AIS keys through .env.
2026-05-02 20:18:46 -06:00
BigBodyCobain 8d3c7a51b7 Fix Docker frontend hydration under CSP
Render the app shell dynamically so Next can attach per-request CSP nonces to its production scripts, preventing Docker from serving a static shell that cannot hydrate. Also gives the first-contact warmup test enough time in CI.
2026-05-02 19:47:32 -06:00
BigBodyCobain fa18c032e2 Fix Docker first-run startup data seeding
Seed safe static backend data into fresh Docker volumes, tighten Docker build-context exclusions, avoid optional env warnings, and make the frontend healthcheck use the IPv4 loopback path that works inside the container.
2026-05-02 19:27:59 -06:00
BigBodyCobain e1060193d0 Improve v0.9.7 startup and runtime reliability
Prioritize cached first-paint data, defer heavyweight feed synthesis, make MeshChat activation explicit, improve CCTV media handling, and tighten desktop runtime packaging filters.
2026-05-02 17:31:54 -06:00
BigBodyCobain 08810f2537 fix: stabilize v0.9.7 startup and feeds 2026-05-02 13:35:49 -06:00
BigBodyCobain f5b9d14b48 Merge remote-tracking branch 'origin/main' 2026-05-02 09:40:23 -06:00
BigBodyCobain 9122d306cd fix: refresh privacy-core pin on source startup 2026-05-02 09:38:13 -06:00
Shadowbroker 03e5fc1363 Update README.md 2026-05-02 09:20:40 -06:00
BigBodyCobain 447afe0b2b build: refresh v0.9.7 updater key 2026-05-02 02:24:46 -06:00
BigBodyCobain d515aba450 fix: polish v0.9.7 micro update 2026-05-02 02:13:36 -06:00
Shadowbroker 3a8db7f9cd Update README.md 2026-05-02 00:30:34 -06:00
Shadowbroker f1cb1e860d Update README.md 2026-05-02 00:30:15 -06:00
Shadowbroker 38bcc976a4 Merge pull request #140 from BigBodyCobain/dependabot/pip/backend/yfinance-1.3.0
Upgrades yfinance from 0.2.54 to 1.3.0 in /backend
2026-05-02 00:26:10 -06:00
Shadowbroker 77b4361ad6 Merge pull request #141 from BigBodyCobain/dependabot/pip/backend/playwright-1.59.0
Bump playwright from 1.50.0 to 1.59.0 in /backend
2026-05-02 00:25:23 -06:00
Shadowbroker c5819d40d1 Merge pull request #138 from BigBodyCobain/dependabot/pip/backend/pydantic-2.13.3
Gets pydantic from 2.11.1 to 2.13.3 in /backend
2026-05-02 00:24:54 -06:00
Shadowbroker 009574db81 Merge pull request #143 from BigBodyCobain/dependabot/pip/backend/sgp4-2.25
Updates sgp4 from 2.23 to 2.25 in /backend
2026-05-02 00:24:32 -06:00
Shadowbroker 281371e135 Merge pull request #145 from BigBodyCobain/dependabot/npm_and_yarn/frontend/eslint-config-next-16.2.4
Upgrades eslint-config-next from 16.1.6 to 16.2.4 in /frontend
2026-05-02 00:24:02 -06:00
Shadowbroker 401268f22a Merge pull request #142 from BigBodyCobain/dependabot/npm_and_yarn/frontend/tailwindcss/postcss-4.2.4
Bumps @tailwindcss/postcss from 4.2.1 to 4.2.4 in /frontend
2026-05-02 00:23:25 -06:00
Shadowbroker f830148e69 Merge pull request #144 from BigBodyCobain/dependabot/npm_and_yarn/frontend/prettier-3.8.3
bump prettier from 3.8.1 to 3.8.3 in /frontend
2026-05-02 00:22:50 -06:00
Shadowbroker 4068c31cfa Update README.md 2026-05-02 00:17:45 -06:00
Shadowbroker 50721816fa Merge pull request #148 from BigBodyCobain/codex/v0.9.7-postmerge-ci
test: stabilize v0.9.7 post-merge CI
2026-05-02 00:01:59 -06:00
BigBodyCobain 5dac844532 test: stabilize secure mail warmup assertion 2026-05-01 23:54:25 -06:00
dependabot[bot] 8884675845 chore(deps-dev): bump eslint-config-next in /frontend
Bumps [eslint-config-next](https://github.com/vercel/next.js/tree/HEAD/packages/eslint-config-next) from 16.1.6 to 16.2.4.
- [Release notes](https://github.com/vercel/next.js/releases)
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js)
- [Commits](https://github.com/vercel/next.js/commits/v16.2.4/packages/eslint-config-next)

---
updated-dependencies:
- dependency-name: eslint-config-next
  dependency-version: 16.2.4
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-02 05:49:22 +00:00
dependabot[bot] 58144d1b82 chore(deps-dev): bump prettier from 3.8.1 to 3.8.3 in /frontend
Bumps [prettier](https://github.com/prettier/prettier) from 3.8.1 to 3.8.3.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prettier/prettier/compare/3.8.1...3.8.3)

---
updated-dependencies:
- dependency-name: prettier
  dependency-version: 3.8.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-02 05:49:08 +00:00
dependabot[bot] da2a27f92a chore(deps): bump sgp4 from 2.23 to 2.25 in /backend
Bumps [sgp4](https://github.com/brandon-rhodes/python-sgp4) from 2.23 to 2.25.
- [Commits](https://github.com/brandon-rhodes/python-sgp4/compare/2.23...2.25)

---
updated-dependencies:
- dependency-name: sgp4
  dependency-version: '2.25'
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-02 05:49:04 +00:00
dependabot[bot] f6f6176a12 chore(deps-dev): bump @tailwindcss/postcss in /frontend
Bumps [@tailwindcss/postcss](https://github.com/tailwindlabs/tailwindcss/tree/HEAD/packages/@tailwindcss-postcss) from 4.2.1 to 4.2.4.
- [Release notes](https://github.com/tailwindlabs/tailwindcss/releases)
- [Changelog](https://github.com/tailwindlabs/tailwindcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/tailwindlabs/tailwindcss/commits/v4.2.4/packages/@tailwindcss-postcss)

---
updated-dependencies:
- dependency-name: "@tailwindcss/postcss"
  dependency-version: 4.2.4
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-02 05:49:02 +00:00
dependabot[bot] e6bea9dad3 chore(deps): bump playwright from 1.50.0 to 1.59.0 in /backend
Bumps [playwright](https://github.com/microsoft/playwright-python) from 1.50.0 to 1.59.0.
- [Release notes](https://github.com/microsoft/playwright-python/releases)
- [Commits](https://github.com/microsoft/playwright-python/compare/v1.50.0...v1.59.0)

---
updated-dependencies:
- dependency-name: playwright
  dependency-version: 1.59.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-02 05:49:00 +00:00
dependabot[bot] aebd5f0198 chore(deps): bump yfinance from 0.2.54 to 1.3.0 in /backend
Bumps [yfinance](https://github.com/ranaroussi/yfinance) from 0.2.54 to 1.3.0.
- [Release notes](https://github.com/ranaroussi/yfinance/releases)
- [Changelog](https://github.com/ranaroussi/yfinance/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/ranaroussi/yfinance/compare/0.2.54...1.3.0)

---
updated-dependencies:
- dependency-name: yfinance
  dependency-version: 1.3.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-02 05:48:56 +00:00
dependabot[bot] 2f70b50f65 chore(deps): bump pydantic from 2.11.1 to 2.13.3 in /backend
Bumps [pydantic](https://github.com/pydantic/pydantic) from 2.11.1 to 2.13.3.
- [Release notes](https://github.com/pydantic/pydantic/releases)
- [Changelog](https://github.com/pydantic/pydantic/blob/main/HISTORY.md)
- [Commits](https://github.com/pydantic/pydantic/compare/v2.11.1...v2.13.3)

---
updated-dependencies:
- dependency-name: pydantic
  dependency-version: 2.13.3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-02 05:48:49 +00:00
Shadowbroker 1b2ad5023d Merge pull request #137 from BigBodyCobain/codex/v0.9.7-release
release: prepare v0.9.7
2026-05-01 23:47:58 -06:00
102 changed files with 5777 additions and 1448 deletions
+33
View File
@@ -8,6 +8,18 @@ __pycache__/
venv/ venv/
.venv/ .venv/
.ruff_cache/ .ruff_cache/
local-artifacts/
release-secrets/
# Never send local configuration or credentials into Docker builds
.env
.env.*
**/.env
**/.env.*
*.pem
*.key
*.p12
*.pfx
# privacy-core build caches (source is needed, artifacts are not) # privacy-core build caches (source is needed, artifacts are not)
privacy-core/target/ privacy-core/target/
@@ -21,3 +33,24 @@ privacy-core/.codex-tmp/
*.log *.log
extra/ extra/
prototype/ prototype/
# Runtime state generated by local backend runs
backend/.pytest_cache/
backend/.ruff_cache/
backend/backend.egg-info/
backend/build/
backend/node_modules/
backend/timemachine/
backend/venv/
backend/data/*cache*.json
backend/data/**/*cache*.json
backend/data/wormhole*.json
backend/data/**/wormhole*.json
backend/data/dm_*.json
backend/data/**/dm_*.json
backend/data/**/peer_store.json
backend/data/**/node.json
backend/data/*.log
backend/data/**/*.log
backend/data/*.key
backend/data/**/*.key
+29
View File
@@ -38,6 +38,35 @@ ADMIN_KEY=
# Leave blank to skip this optional enrichment. # Leave blank to skip this optional enrichment.
# NUFORC_MAPBOX_TOKEN= # NUFORC_MAPBOX_TOKEN=
# Optional startup-risk controls.
# On Windows, external curl fallback and the Playwright LiveUAMap scraper are
# disabled by default so blocked upstream feeds cannot interrupt start.bat.
# SHADOWBROKER_ENABLE_WINDOWS_CURL_FALLBACK=false
# SHADOWBROKER_ENABLE_LIVEUAMAP_SCRAPER=false
# AIS starts by default when AIS_API_KEY is set. Set to 0/false to force-disable.
# SHADOWBROKER_ENABLE_AIS_STREAM_PROXY=true
# Minimum visible satellite catalog before forcing a CelesTrak refresh.
# SHADOWBROKER_MIN_VISIBLE_SATELLITES=350
# Upper bound for TLE fallback satellite search when CelesTrak is unreachable.
# SHADOWBROKER_MAX_VISIBLE_SATELLITES=450
# NUFORC fallback uses the Hugging Face mirror when live NUFORC is unavailable.
# NUFORC_HF_FALLBACK_LIMIT=250
# NUFORC_HF_GEOCODE_LIMIT=150
# First-paint cache age budgets. These let the map and Global Threat Intercept
# paint from the last local snapshot while live feeds refresh in the background.
# FAST_STARTUP_CACHE_MAX_AGE_S=21600
# INTEL_STARTUP_CACHE_MAX_AGE_S=21600
# Docker resource tuning. The backend synthesizes large geospatial feeds; keep
# this at 4G or higher on hosts that run AIS, OpenSky, CCTV, satellites, and
# threat feeds together. Lower caps can cause Docker OOM restarts and empty
# slow layers such as news, UAP sightings, military bases, and wastewater.
# BACKEND_MEMORY_LIMIT=4G
# SHADOWBROKER_FETCH_WORKERS=8
# SHADOWBROKER_SLOW_FETCH_CONCURRENCY=4
# SHADOWBROKER_STARTUP_HEAVY_CONCURRENCY=2
# Google Earth Engine for VIIRS night lights change detection (optional). # Google Earth Engine for VIIRS night lights change detection (optional).
# pip install earthengine-api # pip install earthengine-api
# GEE_SERVICE_ACCOUNT_KEY= # GEE_SERVICE_ACCOUNT_KEY=
+50 -16
View File
@@ -11,15 +11,15 @@
![ShadowBroker](/uploads/46f99d19fa141a2efba37feee9de8aab/Title.jpg) [![ShadowBroker](/uploads/46f99d19fa141a2efba37feee9de8aab/Title.jpg)](https://github.com/user-attachments/assets/248208ec-62f7-49d1-831d-4bd0a1fa6852)
**ShadowBroker** is a decentralized real-time, multi-domain OSINT dashboard that fuses 60+ live intelligence feeds into a single dark-ops map interface. Aircraft, ships, satellites, conflict zones, CCTV networks, GPS jamming, internet-connected devices, police scanners, mesh radio nodes, and breaking geopolitical events — all updating in real time on one screen as well as a obfuscated communications protocol and information exchange infrastructure. **ShadowBroker** is a decentralized intelligence platform that aggregates real-time, multi-domain OSINT telemetry from 60+ live intelligence feeds into a single dark-ops map interface. Aircraft, ships, satellites, conflict zones, CCTV networks, GPS jamming, internet-connected devices, police scanners, mesh radio nodes, and breaking geopolitical events — all updating in real time on one screen as well as an obfuscated communications protocol and information exchange infrastructure.
Built with **Next.js**, **MapLibre GL**, **FastAPI**, and **Python**. 35+ toggleable data layers including SAR ground-change detection. Multiple visual modes (DEFAULT / SATELLITE / FLIR / NVG / CRT). Right-click any point on Earth for a country dossier, head-of-state lookup, and the latest Sentinel-2 satellite photo. No user data is collected or transmitted — the dashboard runs entirely in your browser against a self-hosted backend. Built with **Next.js**, **MapLibre GL**, **FastAPI**, and **Python**. 35+ toggleable data layers, including SAR ground-change detection. Multiple visual modes (DEFAULT / SATELLITE / FLIR / NVG / CRT). Right-click any point on Earth for a country dossier, head-of-state lookup, and the latest Sentinel-2 satellite photo. No user data is collected or transmitted — the dashboard runs entirely in your browser against a self-hosted backend.
Designed for analysts, researchers, radio operators, and anyone who wants to see what the world looks like when every public signal is on the same map. Designed for analysts, researchers, radio operators, and anyone who wants to see what the world looks like when every public signal is on the same map.
@@ -38,10 +38,9 @@ ShadowBroker includes an optional Shodan connector for operator-supplied API acc
## Interesting Use Cases ## Interesting Use Cases
* **Track Air Force One**, the private jets of billionaires and dictators, and every military tanker, ISR, and fighter broadcasting ADS-B. Air Force One and all of the accompanying Presidential/Vice Presidential planes are highlighted and monitored from the moment they leave the ground.
* **Connect an AI agent as a co-analyst** through ShadowBroker's HMAC-signed agentic command channel — supports OpenClaw and any other agent that speaks the protocol (Claude, GPT, LangChain, custom). The agent gets full read/write access to all 35+ data layers, pin placement, map control, SAR ground-change, mesh networking, and alert delivery. It sees everything the operator sees and can take actions on the map in real time.
* **Communicate on the InfoNet testnet** — The first decentralized intelligence mesh built into an OSINT tool. Obfuscated messaging with gate personas, Dead Drop peer-to-peer exchange, and a built-in terminal CLI. No accounts, no signup. Privacy is not guaranteed yet — this is an experimental testnet — but the protocol is live and being hardened. * **Communicate on the InfoNet testnet** — The first decentralized intelligence mesh built into an OSINT tool. Obfuscated messaging with gate personas, Dead Drop peer-to-peer exchange, and a built-in terminal CLI. No accounts, no signup. Privacy is not guaranteed yet — this is an experimental testnet — but the protocol is live and being hardened.
* **Track Air Force One**, the private jets of billionaires and dictators, and every military tanker, ISR, and fighter broadcasting ADS-B — with automatic holding pattern detection when aircraft start circling
* **Estimate where US aircraft carriers are** using automated GDELT news scraping — no other open tool does this
* **Search internet-connected devices worldwide** via Shodan — cameras, SCADA systems, databases — plotted as a live overlay on the map
* **Right-click anywhere on Earth** for a country dossier (head of state, population, languages), Wikipedia summary, and the latest Sentinel-2 satellite photo at 10m resolution * **Right-click anywhere on Earth** for a country dossier (head of state, population, languages), Wikipedia summary, and the latest Sentinel-2 satellite photo at 10m resolution
* **Click a KiwiSDR node** and tune into live shortwave radio directly in the dashboard. Click a police scanner feed and eavesdrop in one click. * **Click a KiwiSDR node** and tune into live shortwave radio directly in the dashboard. Click a police scanner feed and eavesdrop in one click.
* **Watch 11,000+ CCTV cameras** across 6 countries — London, NYC, California, Spain, Singapore, and more — streaming live on the map * **Watch 11,000+ CCTV cameras** across 6 countries — London, NYC, California, Spain, Singapore, and more — streaming live on the map
@@ -51,10 +50,12 @@ ShadowBroker includes an optional Shodan connector for operator-supplied API acc
* **Follow earthquakes, volcanic eruptions, active wildfires** (NASA FIRMS), severe weather alerts, and air quality readings worldwide * **Follow earthquakes, volcanic eruptions, active wildfires** (NASA FIRMS), severe weather alerts, and air quality readings worldwide
* **Map military bases, 35,000+ power plants**, 2,000+ data centers, and internet outage regions — cross-referenced automatically * **Map military bases, 35,000+ power plants**, 2,000+ data centers, and internet outage regions — cross-referenced automatically
* **Connect to Meshtastic mesh radio nodes** and APRS amateur radio networks — visible on the map and integrated into Mesh Chat * **Connect to Meshtastic mesh radio nodes** and APRS amateur radio networks — visible on the map and integrated into Mesh Chat
* **Connect an AI agent as a co-analyst** through ShadowBroker's HMAC-signed agentic command channel — supports OpenClaw and any other agent that speaks the protocol (Claude, GPT, LangChain, custom). The agent gets full read/write access to all 35+ data layers, pin placement, map control, SAR ground-change, mesh networking, and alert delivery. It sees everything the operator sees and can take actions on the map in real time.
* **Detect ground changes through cloud cover** with SAR (Synthetic Aperture Radar) — mm-scale ground deformation, flood extent, vegetation disturbance, and damage assessments from NASA OPERA and Copernicus EGMS. Define your own watch areas and get anomaly alerts. Free with a NASA Earthdata account. * **Detect ground changes through cloud cover** with SAR (Synthetic Aperture Radar) — mm-scale ground deformation, flood extent, vegetation disturbance, and damage assessments from NASA OPERA and Copernicus EGMS. Define your own watch areas and get anomaly alerts. Free with a NASA Earthdata account.
* **Switch visual modes** — DEFAULT, SATELLITE, FLIR (thermal), NVG (night vision), CRT (retro terminal) — via the STYLE button * **Switch visual modes** — DEFAULT, SATELLITE, FLIR (thermal), NVG (night vision), CRT (retro terminal) — via the STYLE button
* **Track trains** across the US (Amtrak) and Europe (DigiTraffic) in real time * **Track trains** across the US (Amtrak) and Europe (DigiTraffic) in real time
* **Estimate where US aircraft carriers are** using automated GDELT news scraping — no other open tool does this
* **Search internet-connected devices worldwide** via Shodan — cameras, SCADA systems, databases — plotted as a live overlay on the map
--- ---
@@ -69,7 +70,11 @@ docker compose up -d
Open `http://localhost:3000` to view the dashboard! *(Requires [Docker Desktop](https://www.docker.com/products/docker-desktop/) or Docker Engine)* Open `http://localhost:3000` to view the dashboard! *(Requires [Docker Desktop](https://www.docker.com/products/docker-desktop/) or Docker Engine)*
> **Podman users:** Replace `docker compose` with `podman compose`, or use the `compose.sh` wrapper which auto-detects your engine. Force Podman with `./compose.sh --engine podman up -d`. > **Backend port already in use?** The browser only needs port `3000`, but the backend API is also published on host port `8000` for local diagnostics. If another app already uses `8000`, create or edit `.env` next to `docker-compose.yml` and set `BACKEND_PORT=8001`, then run `docker compose up -d`.
> **Blank news/UAP/bases/wastewater after several minutes?** Check for backend OOM restarts with `docker events --since 30m --filter container=shadowbroker-backend --filter event=oom`. The default compose file gives the backend 4GB; if your host has less memory, reduce enabled feeds or set `BACKEND_MEMORY_LIMIT=3G` and expect slower/heavier layers to warm more gradually.
> **Podman users:** Podman works, but `podman compose` is a wrapper and still needs a Compose provider installed. On Windows/WSL, if you see `looking up compose provider failed`, install `podman-compose` and run `podman-compose pull` followed by `podman-compose up -d` from inside the cloned `Shadowbroker` folder. On Linux/macOS/WSL shells you can also use `./compose.sh --engine podman pull` and `./compose.sh --engine podman up -d`.
--- ---
@@ -92,6 +97,8 @@ That's it. `pull` grabs the latest images, `up -d` restarts the containers.
> docker compose pull > docker compose pull
> docker compose up -d > docker compose up -d
> ``` > ```
>
> Podman users should run the equivalent provider command, for example `podman-compose pull` and `podman-compose up -d`, or use `./compose.sh --engine podman pull` and `./compose.sh --engine podman up -d` from a bash-compatible shell.
### ⚠️ **Stuck on the old version?** ### ⚠️ **Stuck on the old version?**
@@ -559,25 +566,51 @@ Open `http://localhost:3000` to view the dashboard.
> **Deploying publicly or on a LAN?** No configuration needed for most setups. > **Deploying publicly or on a LAN?** No configuration needed for most setups.
> The frontend proxies all API calls through the Next.js server to `BACKEND_URL`, > The frontend proxies all API calls through the Next.js server to `BACKEND_URL`,
> which defaults to `http://backend:8000` (Docker internal networking). > which defaults to `http://backend:8000` (Docker internal networking).
> Port 8000 does not need to be exposed externally. > Host port `8000` is only published for local API/debug access. If it conflicts
> with another service, set `BACKEND_PORT=8001` in `.env`; leave `BACKEND_URL`
> as `http://backend:8000` because that is the Docker-internal port.
> The backend memory cap is controlled by `BACKEND_MEMORY_LIMIT` and defaults
> to `4G`. If Docker reports OOM events, the backend will restart and slow
> layers can look empty until they repopulate.
> >
> If your backend runs on a **different host or port**, set `BACKEND_URL` at runtime — no rebuild required: > If your backend runs on a **different host or port**, set `BACKEND_URL` at runtime — no rebuild required:
> >
> ```bash > ```bash
> # Linux / macOS > # Linux / macOS
> BACKEND_URL=http://myserver.com:9096 docker-compose up -d > BACKEND_URL=http://myserver.com:9096 docker compose up -d
> >
> # Podman (via compose.sh wrapper) > # Podman (via compose.sh wrapper)
> BACKEND_URL=http://192.168.1.50:9096 ./compose.sh up -d > BACKEND_URL=http://192.168.1.50:9096 ./compose.sh up -d
> >
> # Windows (PowerShell) > # Windows (PowerShell)
> $env:BACKEND_URL="http://myserver.com:9096"; docker-compose up -d > $env:BACKEND_URL="http://myserver.com:9096"; docker compose up -d
> >
> # Or add to a .env file next to docker-compose.yml: > # Or add to a .env file next to docker-compose.yml:
> # BACKEND_URL=http://myserver.com:9096 > # BACKEND_URL=http://myserver.com:9096
> ``` > ```
**Podman users:** Replace `docker compose` with `podman compose`, or use the `compose.sh` wrapper which auto-detects your engine. **Podman users:** Do not pass the GitHub URL to `podman compose pull`; clone the repo first, `cd Shadowbroker`, then run compose from that folder. `podman compose` also requires a Compose provider. If Podman reports `looking up compose provider failed`, install one:
```bash
# Linux / macOS / WSL
python3 -m pip install --user podman-compose
podman-compose pull
podman-compose up -d
```
```powershell
# Windows PowerShell
py -m pip install --user podman-compose
podman-compose pull
podman-compose up -d
```
If you are in a bash-compatible shell, the included wrapper can auto-detect Docker or Podman:
```bash
./compose.sh --engine podman pull
./compose.sh --engine podman up -d
```
--- ---
@@ -599,7 +632,7 @@ services:
# image: registry.gitlab.com/bigbodycobain/shadowbroker/backend:latest # image: registry.gitlab.com/bigbodycobain/shadowbroker/backend:latest
container_name: shadowbroker-backend container_name: shadowbroker-backend
ports: ports:
- "8000:8000" - "${BACKEND_PORT:-8000}:8000"
environment: environment:
- AIS_API_KEY=your_aisstream_key # Required — get one free at aisstream.io - AIS_API_KEY=your_aisstream_key # Required — get one free at aisstream.io
- OPENSKY_CLIENT_ID= # Optional — higher flight data rate limits - OPENSKY_CLIENT_ID= # Optional — higher flight data rate limits
@@ -629,7 +662,7 @@ volumes:
backend_data: backend_data:
``` ```
> **How it works:** The frontend container proxies all `/api/*` requests through the Next.js server to `BACKEND_URL` using Docker's internal networking. The browser only ever talks to port 3000 — port 8000 does not need to be exposed externally. > **How it works:** The frontend container proxies all `/api/*` requests through the Next.js server to `BACKEND_URL` using Docker's internal networking. The browser only ever talks to port 3000. The backend's host port is for local API/debug access and can be changed with `BACKEND_PORT=8001` without changing `BACKEND_URL`.
> >
> `BACKEND_URL` is a plain runtime environment variable (not a build-time `NEXT_PUBLIC_*`), so you can change it in Portainer, Uncloud, or any compose editor without rebuilding the image. Set it to the address where your backend is reachable from inside the Docker network (e.g. `http://backend:8000`, `http://192.168.1.50:8000`). > `BACKEND_URL` is a plain runtime environment variable (not a build-time `NEXT_PUBLIC_*`), so you can change it in Portainer, Uncloud, or any compose editor without rebuilding the image. Set it to the address where your backend is reachable from inside the Docker network (e.g. `http://backend:8000`, `http://192.168.1.50:8000`).
@@ -932,8 +965,9 @@ Then confirm authenticated `GET /api/wormhole/status` or `GET /api/settings/worm
| Variable | Where to set | Purpose | | Variable | Where to set | Purpose |
|---|---|---| |---|---|---|
| `BACKEND_URL` | `environment` in `docker-compose.yml`, or shell env | URL the Next.js server uses to proxy API calls to the backend. Defaults to `http://backend:8000`. **Runtime variable — no rebuild needed.** | | `BACKEND_URL` | `environment` in `docker-compose.yml`, or shell env | URL the Next.js server uses to proxy API calls to the backend. Defaults to `http://backend:8000`. **Runtime variable — no rebuild needed.** |
| `BACKEND_PORT` | repo-root `.env` or shell env before `docker compose up` | Host port used to expose the backend API for local diagnostics. Defaults to `8000`; set `BACKEND_PORT=8001` if port 8000 is already in use. Does not change Docker-internal `BACKEND_URL`. |
**How it works:** The frontend proxies all `/api/*` requests through the Next.js server to `BACKEND_URL` using Docker's internal networking. Browsers only talk to port 3000; port 8000 never needs to be exposed externally. For local dev without Docker, `BACKEND_URL` defaults to `http://localhost:8000`. **How it works:** The frontend proxies all `/api/*` requests through the Next.js server to `BACKEND_URL` using Docker's internal networking. Browsers only talk to port 3000; the backend host port is only for local diagnostics. For local dev without Docker, `BACKEND_URL` defaults to `http://localhost:8000`.
--- ---
@@ -943,7 +977,7 @@ ShadowBroker is built in the open. These people shipped real code:
| Who | What | PR | | Who | What | PR |
|-----|------|----| |-----|------|----|
| [@Alienmajik](https://github.com/Alienmajik) | Raspberry Pi 5 support — ARM64 packaging, headless deployment notes, runtime tuning for Pi-class hardware | — | | [@Alienmajik](https://gitlab.com/Alienmajik) | Raspberry Pi 5 support — ARM64 packaging, headless deployment notes, runtime tuning for Pi-class hardware | — |
| [@wa1id](https://github.com/wa1id) | CCTV ingestion fix — threaded SQLite, persistent DB, startup hydration, cluster clickability | #92 | | [@wa1id](https://github.com/wa1id) | CCTV ingestion fix — threaded SQLite, persistent DB, startup hydration, cluster clickability | #92 |
| [@AlborzNazari](https://github.com/AlborzNazari) | Spain DGT + Madrid CCTV sources, STIX 2.1 threat intel export | #91 | | [@AlborzNazari](https://github.com/AlborzNazari) | Spain DGT + Madrid CCTV sources, STIX 2.1 threat intel export | #91 |
| [@adust09](https://github.com/adust09) | Power plants layer, East Asia intel coverage (JSDF bases, ICAO enrichment, Taiwan news, military classification) | #71, #72, #76, #77, #87 | | [@adust09](https://github.com/adust09) | Power plants layer, East Asia intel coverage (JSDF bases, ICAO enrichment, Taiwan news, military classification) | #71, #72, #76, #77, #87 |
+9 -3
View File
@@ -54,8 +54,9 @@ AIS_API_KEY= # https://aisstream.io/ — free tier WebSocket key
# MESH_MQTT_INCLUDE_DEFAULT_ROOTS=true # MESH_MQTT_INCLUDE_DEFAULT_ROOTS=true
# MESH_MQTT_BROKER=mqtt.meshtastic.org # MESH_MQTT_BROKER=mqtt.meshtastic.org
# MESH_MQTT_PORT=1883 # MESH_MQTT_PORT=1883
# MESH_MQTT_USER=meshdev # Leave user/pass blank for the public Meshtastic broker default.
# MESH_MQTT_PASS=large4cats # MESH_MQTT_USER=
# MESH_MQTT_PASS=
# Optional Meshtastic node ID (e.g. "!abcd1234"). When set, included in the # Optional Meshtastic node ID (e.g. "!abcd1234"). When set, included in the
# User-Agent sent to meshtastic.liamcottle.net so the upstream service operator # User-Agent sent to meshtastic.liamcottle.net so the upstream service operator
@@ -127,7 +128,12 @@ AIS_API_KEY= # https://aisstream.io/ — free tier WebSocket key
# MESH_BOOTSTRAP_DISABLED=false # MESH_BOOTSTRAP_DISABLED=false
# MESH_BOOTSTRAP_MANIFEST_PATH=data/bootstrap_peers.json # MESH_BOOTSTRAP_MANIFEST_PATH=data/bootstrap_peers.json
# MESH_BOOTSTRAP_SIGNER_PUBLIC_KEY= # MESH_BOOTSTRAP_SIGNER_PUBLIC_KEY=
# MESH_DEFAULT_SYNC_PEERS=https://node.shadowbroker.info # bundled pull-only public seed for fresh installs # Infonet/Wormhole fails closed to onion/RNS by default. Only enable clearnet
# sync for local relay development or an explicitly public testnet.
# MESH_INFONET_ALLOW_CLEARNET_SYNC=false
# MESH_BOOTSTRAP_SEED_PEERS=http://gqpbunqbgtkcqilvclm3xrkt3zowjyl3s62kkktvojgvxzizamvbrqid.onion:8000
# Add comma-separated http://*.onion peers as more private seed/relay nodes come online.
# MESH_DEFAULT_SYNC_PEERS= # legacy alias; prefer MESH_BOOTSTRAP_SEED_PEERS
# MESH_RELAY_PEERS= # comma-separated operator-trusted sync/push peers (empty by default) # MESH_RELAY_PEERS= # comma-separated operator-trusted sync/push peers (empty by default)
# MESH_PEER_PUSH_SECRET= # REQUIRED when relay/RNS peers are configured (min 16 chars, generate with: python -c "import secrets; print(secrets.token_urlsafe(32))") # MESH_PEER_PUSH_SECRET= # REQUIRED when relay/RNS peers are configured (min 16 chars, generate with: python -c "import secrets; print(secrets.token_urlsafe(32))")
# MESH_SYNC_INTERVAL_S=300 # MESH_SYNC_INTERVAL_S=300
+11 -1
View File
@@ -22,10 +22,12 @@ FROM python:3.11-slim-bookworm
WORKDIR /app WORKDIR /app
# Install Node.js (for AIS WebSocket proxy) and curl (for network fallback) # Install Node.js (for AIS WebSocket proxy), curl (for network fallback), and
# Tor (for Wormhole/remote-agent .onion transport).
RUN apt-get update && apt-get install -y --no-install-recommends \ RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates \ ca-certificates \
curl \ curl \
tor \
&& curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \ && curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \
&& apt-get install -y --no-install-recommends nodejs \ && apt-get install -y --no-install-recommends nodejs \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
@@ -49,6 +51,13 @@ RUN cd /workspace/backend && uv sync --frozen --no-dev \
# Copy backend source code # Copy backend source code
COPY backend/ . COPY backend/ .
# Preserve safe static data outside /app/data. The compose named volume mounted
# at /app/data hides image-baked files on first run, so the entrypoint seeds
# missing static JSON into fresh volumes before the backend starts.
RUN mkdir -p /app/image-data \
&& if [ -d /app/data ]; then cp -a /app/data/. /app/image-data/; fi \
&& chmod +x /app/docker-entrypoint.sh
# Install Node.js dependencies (ws module for AIS WebSocket proxy) # Install Node.js dependencies (ws module for AIS WebSocket proxy)
COPY backend/package*.json ./ COPY backend/package*.json ./
RUN npm ci --omit=dev RUN npm ci --omit=dev
@@ -75,4 +84,5 @@ USER backenduser
EXPOSE 8000 EXPOSE 8000
# Start FastAPI server # Start FastAPI server
ENTRYPOINT ["/app/docker-entrypoint.sh"]
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000", "--timeout-keep-alive", "120"] CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000", "--timeout-keep-alive", "120"]
+38 -5
View File
@@ -13,6 +13,7 @@ import hmac
import asyncio import asyncio
import hmac as _hmac_mod import hmac as _hmac_mod
import hashlib as _hashlib_mod import hashlib as _hashlib_mod
import ipaddress
import json as json_mod import json as json_mod
import logging import logging
import time import time
@@ -235,10 +236,36 @@ def _is_local_or_docker(host: str) -> bool:
return host in {"127.0.0.1", "::1", "localhost"} return host in {"127.0.0.1", "::1", "localhost"}
def _docker_bridge_local_operator_enabled() -> bool:
return str(os.environ.get("SHADOWBROKER_TRUST_DOCKER_BRIDGE_LOCAL_OPERATOR", "")).strip().lower() in {
"1",
"true",
"yes",
"on",
}
def _is_docker_bridge_host(host: str) -> bool:
try:
ip = ipaddress.ip_address(host)
except ValueError:
return False
# Docker Desktop and the default compose bridge normally sit inside
# 172.16.0.0/12. Keep this narrower than "any private IP" so a user who
# intentionally binds the backend to LAN does not silently trust LAN clients.
return ip in ipaddress.ip_network("172.16.0.0/12")
def _is_trusted_local_runtime_host(host: str) -> bool:
if _is_local_or_docker(host):
return True
return _docker_bridge_local_operator_enabled() and _is_docker_bridge_host(host)
def require_local_operator(request: Request): def require_local_operator(request: Request):
"""Allow local tooling on loopback / Docker internal network, or a valid admin key.""" """Allow local tooling on loopback / Docker internal network, or a valid admin key."""
host = (request.client.host or "").lower() if request.client else "" host = (request.client.host or "").lower() if request.client else ""
if _is_local_or_docker(host) or (_debug_mode_enabled() and host == "test"): if _is_trusted_local_runtime_host(host) or (_debug_mode_enabled() and host == "test"):
return return
admin_key = _current_admin_key() admin_key = _current_admin_key()
presented = str(request.headers.get("X-Admin-Key", "") or "").strip() presented = str(request.headers.get("X-Admin-Key", "") or "").strip()
@@ -362,8 +389,8 @@ async def require_openclaw_or_local(request: Request):
""" """
host = (request.client.host or "").lower() if request.client else "" host = (request.client.host or "").lower() if request.client else ""
# 1. Local loopback — always allowed # 1. Local runtime path — loopback, plus bundled Docker bridge when compose opts in
if _is_local_or_docker(host) or (_debug_mode_enabled() and host == "test"): if _is_trusted_local_runtime_host(host) or (_debug_mode_enabled() and host == "test"):
return return
# 2. Admin key — full trust # 2. Admin key — full trust
@@ -402,7 +429,9 @@ async def require_openclaw_or_local(request: Request):
# Startup validators # Startup validators
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
_KNOWN_COMPROMISED_PEER_PUSH_SECRET = "Mv63UvLfwqOEVWeRBXjA8MtFl2nEkkhUlLYVHiX1Zzo" _KNOWN_COMPROMISED_PEER_PUSH_SECRET_SHA256 = (
"be05bc75350d6e5d2e154e969c4dfc14bab1e48a9661c64ab7a331e0aa96aea7"
)
def _validate_admin_startup() -> None: def _validate_admin_startup() -> None:
@@ -492,7 +521,11 @@ def _validate_peer_push_secret() -> None:
secret = os.environ.get("MESH_PEER_PUSH_SECRET", "").strip() secret = os.environ.get("MESH_PEER_PUSH_SECRET", "").strip()
# Replace the known-compromised testnet default automatically # Replace the known-compromised testnet default automatically
if secret == _KNOWN_COMPROMISED_PEER_PUSH_SECRET: if (
secret
and _hashlib_mod.sha256(secret.encode("utf-8")).hexdigest()
== _KNOWN_COMPROMISED_PEER_PUSH_SECRET_SHA256
):
logger.warning( logger.warning(
"MESH_PEER_PUSH_SECRET was the publicly-known testnet default — " "MESH_PEER_PUSH_SECRET was the publicly-known testnet default — "
"auto-generating a secure replacement." "auto-generating a secure replacement."
+20 -10
View File
@@ -1,15 +1,5 @@
{ {
"feeds": [ "feeds": [
{
"name": "Reuters",
"url": "https://www.reutersagency.com/feed/?best-topics=world",
"weight": 5
},
{
"name": "AP News",
"url": "https://rsshub.app/apnews/topics/world-news",
"weight": 5
},
{ {
"name": "NPR", "name": "NPR",
"url": "https://feeds.npr.org/1004/rss.xml", "url": "https://feeds.npr.org/1004/rss.xml",
@@ -99,6 +89,26 @@
"name": "Japan Times", "name": "Japan Times",
"url": "https://www.japantimes.co.jp/feed/", "url": "https://www.japantimes.co.jp/feed/",
"weight": 3 "weight": 3
},
{
"name": "CSM",
"url": "https://www.csmonitor.com/rss/world",
"weight": 4
},
{
"name": "PBS NewsHour",
"url": "https://www.pbs.org/newshour/feeds/rss/world",
"weight": 4
},
{
"name": "France 24",
"url": "https://www.france24.com/en/rss",
"weight": 4
},
{
"name": "DW",
"url": "https://rss.dw.com/xml/rss-en-world",
"weight": 4
} }
] ]
} }
+17
View File
@@ -0,0 +1,17 @@
#!/bin/sh
set -eu
# Docker named volumes hide files that were baked into /app/data at image build
# time. Seed safe, static data into a fresh volume so first-run Docker installs
# behave like source installs without bundling local runtime secrets.
if [ -d /app/image-data ]; then
mkdir -p /app/data
find /app/image-data -mindepth 1 -maxdepth 1 -type f | while IFS= read -r src; do
dest="/app/data/$(basename "$src")"
if [ ! -e "$dest" ]; then
cp "$src" "$dest" || true
fi
done
fi
exec "$@"
+338 -50
View File
@@ -14,7 +14,7 @@ from dataclasses import dataclass, field
from typing import Any from typing import Any
from json import JSONDecodeError from json import JSONDecodeError
APP_VERSION = "0.9.7" APP_VERSION = "0.9.75"
logging.basicConfig(level=logging.INFO) logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -200,6 +200,7 @@ from services.data_fetcher import (
start_scheduler, start_scheduler,
stop_scheduler, stop_scheduler,
get_latest_data, get_latest_data,
seed_startup_caches,
) )
from services.ais_stream import start_ais_stream, stop_ais_stream from services.ais_stream import start_ais_stream, stop_ais_stream
from services.carrier_tracker import start_carrier_tracker, stop_carrier_tracker from services.carrier_tracker import start_carrier_tracker, stop_carrier_tracker
@@ -1083,6 +1084,7 @@ _WORMHOLE_PUBLIC_PROFILE_FIELDS = {"profile", "wormhole_enabled"}
_PRIVATE_LANE_CONTROL_FIELDS = {"private_lane_tier", "private_lane_policy"} _PRIVATE_LANE_CONTROL_FIELDS = {"private_lane_tier", "private_lane_policy"}
_PUBLIC_RNS_STATUS_FIELDS = {"enabled", "ready", "configured_peers", "active_peers"} _PUBLIC_RNS_STATUS_FIELDS = {"enabled", "ready", "configured_peers", "active_peers"}
_NODE_PUBLIC_EVENT_HOOK_REGISTERED = False _NODE_PUBLIC_EVENT_HOOK_REGISTERED = False
_INFONET_PRIVATE_TRANSPORT_LOCK = threading.Lock()
def _current_node_mode() -> str: def _current_node_mode() -> str:
@@ -1113,9 +1115,10 @@ def _participant_node_enabled() -> bool:
def _node_runtime_snapshot() -> dict[str, Any]: def _node_runtime_snapshot() -> dict[str, Any]:
with _NODE_RUNTIME_LOCK: with _NODE_RUNTIME_LOCK:
return { return {
"node_mode": _NODE_BOOTSTRAP_STATE.get("node_mode", "participant"), "node_mode": _current_node_mode(),
"node_enabled": _participant_node_enabled(), "node_enabled": _participant_node_enabled(),
"bootstrap": dict(_NODE_BOOTSTRAP_STATE), "private_transport_required": _infonet_private_transport_required(),
"bootstrap": {**dict(_NODE_BOOTSTRAP_STATE), "node_mode": _current_node_mode()},
"sync_runtime": get_sync_state().to_dict(), "sync_runtime": get_sync_state().to_dict(),
"push_runtime": dict(_NODE_PUSH_STATE), "push_runtime": dict(_NODE_PUSH_STATE),
} }
@@ -1148,6 +1151,79 @@ def _set_participant_node_enabled(enabled: bool) -> dict[str, Any]:
} }
def _infonet_private_transport_required() -> bool:
return not bool(getattr(get_settings(), "MESH_INFONET_ALLOW_CLEARNET_SYNC", False))
def _infonet_private_transport_error() -> str:
return "private Infonet requires onion/RNS transport; no clearnet sync fallback"
def _is_private_infonet_transport(transport: str) -> bool:
return str(transport or "").strip().lower() in {"onion", "rns"}
def _filter_infonet_sync_records(records: list[Any]) -> list[Any]:
if not _infonet_private_transport_required():
return records
return [
record
for record in records
if _is_private_infonet_transport(str(getattr(record, "transport", "") or ""))
]
def _ensure_infonet_private_transport_ready(reason: str = "") -> bool:
"""Warm the local onion transport before private Infonet sync.
Infonet may know about an onion seed before the Wormhole UI is opened. The
sync worker still needs Arti marked enabled and a ready SOCKS listener, so
do that lazily in the worker instead of making users manually open another
panel just to participate in the Infonet.
"""
if not _infonet_private_transport_required():
return True
try:
from services.wormhole_supervisor import _check_arti_ready
if bool(get_settings().MESH_ARTI_ENABLED) and _check_arti_ready():
return True
except Exception:
pass
if not _INFONET_PRIVATE_TRANSPORT_LOCK.acquire(blocking=False):
return False
try:
from routers.ai_intel import _write_env_value
from services.tor_hidden_service import tor_service
from services.wormhole_supervisor import _check_arti_ready
label = f" ({reason})" if reason else ""
logger.info("Infonet private transport warmup starting%s", label)
tor_result = tor_service.start(target_port=8000)
if tor_result.get("ok"):
_write_env_value("MESH_ARTI_ENABLED", "true")
get_settings.cache_clear()
if _check_arti_ready():
logger.info("Infonet private transport ready%s", label)
return True
logger.warning("Infonet private transport warmup incomplete%s: %s", label, tor_result)
return False
except Exception as exc:
logger.warning("Infonet private transport warmup failed: %s", exc)
return False
finally:
_INFONET_PRIVATE_TRANSPORT_LOCK.release()
def _configured_bootstrap_seed_peer_urls() -> list[str]:
settings = get_settings()
primary = str(getattr(settings, "MESH_BOOTSTRAP_SEED_PEERS", "") or "").strip()
legacy = str(getattr(settings, "MESH_DEFAULT_SYNC_PEERS", "") or "").strip()
return parse_configured_relay_peers(primary or legacy)
def _refresh_node_peer_store(*, now: float | None = None) -> dict[str, Any]: def _refresh_node_peer_store(*, now: float | None = None) -> dict[str, Any]:
from services.mesh.mesh_bootstrap_manifest import load_bootstrap_manifest_from_settings from services.mesh.mesh_bootstrap_manifest import load_bootstrap_manifest_from_settings
from services.mesh.mesh_peer_store import ( from services.mesh.mesh_peer_store import (
@@ -1166,14 +1242,17 @@ def _refresh_node_peer_store(*, now: float | None = None) -> dict[str, Any]:
except Exception: except Exception:
store = PeerStore(DEFAULT_PEER_STORE_PATH) store = PeerStore(DEFAULT_PEER_STORE_PATH)
private_transport_required = _infonet_private_transport_required()
operator_peers = configured_relay_peer_urls() operator_peers = configured_relay_peer_urls()
default_sync_peers = parse_configured_relay_peers( bootstrap_seed_peers = _configured_bootstrap_seed_peer_urls()
str(getattr(get_settings(), "MESH_DEFAULT_SYNC_PEERS", "") or "") skipped_clearnet_peers = 0
)
for peer_url in operator_peers: for peer_url in operator_peers:
transport = peer_transport_kind(peer_url) transport = peer_transport_kind(peer_url)
if not transport: if not transport:
continue continue
if private_transport_required and not _is_private_infonet_transport(transport):
skipped_clearnet_peers += 1
continue
store.upsert( store.upsert(
make_sync_peer_record( make_sync_peer_record(
peer_url=peer_url, peer_url=peer_url,
@@ -1194,19 +1273,22 @@ def _refresh_node_peer_store(*, now: float | None = None) -> dict[str, Any]:
) )
operator_peer_set = set(operator_peers) operator_peer_set = set(operator_peers)
for peer_url in default_sync_peers: for peer_url in bootstrap_seed_peers:
if peer_url in operator_peer_set: if peer_url in operator_peer_set:
continue continue
transport = peer_transport_kind(peer_url) transport = peer_transport_kind(peer_url)
if not transport: if not transport:
continue continue
if private_transport_required and not _is_private_infonet_transport(transport):
skipped_clearnet_peers += 1
continue
store.upsert( store.upsert(
make_bootstrap_peer_record( make_bootstrap_peer_record(
peer_url=peer_url, peer_url=peer_url,
transport=transport, transport=transport,
role="seed", role="seed",
label="ShadowBroker default seed", label="ShadowBroker bootstrap seed",
signer_id="shadowbroker-default", signer_id="shadowbroker-bootstrap",
now=timestamp, now=timestamp,
) )
) )
@@ -1216,8 +1298,8 @@ def _refresh_node_peer_store(*, now: float | None = None) -> dict[str, Any]:
transport=transport, transport=transport,
role="seed", role="seed",
source="bundle", source="bundle",
label="ShadowBroker default seed", label="ShadowBroker bootstrap seed",
signer_id="shadowbroker-default", signer_id="shadowbroker-bootstrap",
now=timestamp, now=timestamp,
) )
) )
@@ -1231,6 +1313,9 @@ def _refresh_node_peer_store(*, now: float | None = None) -> dict[str, Any]:
if manifest is not None: if manifest is not None:
for peer in manifest.peers: for peer in manifest.peers:
if private_transport_required and not _is_private_infonet_transport(peer.transport):
skipped_clearnet_peers += 1
continue
store.upsert( store.upsert(
make_bootstrap_peer_record( make_bootstrap_peer_record(
peer_url=peer.peer_url, peer_url=peer.peer_url,
@@ -1253,17 +1338,30 @@ def _refresh_node_peer_store(*, now: float | None = None) -> dict[str, Any]:
) )
) )
if private_transport_required and skipped_clearnet_peers and not bootstrap_error:
bootstrap_error = _infonet_private_transport_error()
store.save() store.save()
bootstrap_records = store.records_for_bucket("bootstrap")
sync_records = store.records_for_bucket("sync")
push_records = store.records_for_bucket("push")
if private_transport_required:
bootstrap_records = [record for record in bootstrap_records if _is_private_infonet_transport(record.transport)]
sync_records = [record for record in sync_records if _is_private_infonet_transport(record.transport)]
push_records = [record for record in push_records if _is_private_infonet_transport(record.transport)]
snapshot = { snapshot = {
"node_mode": mode, "node_mode": mode,
"private_transport_required": private_transport_required,
"skipped_clearnet_peer_count": skipped_clearnet_peers,
"manifest_loaded": manifest is not None, "manifest_loaded": manifest is not None,
"manifest_signer_id": manifest.signer_id if manifest is not None else "", "manifest_signer_id": manifest.signer_id if manifest is not None else "",
"manifest_valid_until": int(manifest.valid_until or 0) if manifest is not None else 0, "manifest_valid_until": int(manifest.valid_until or 0) if manifest is not None else 0,
"bootstrap_peer_count": len(store.records_for_bucket("bootstrap")), "bootstrap_peer_count": len(bootstrap_records),
"sync_peer_count": len(store.records_for_bucket("sync")), "sync_peer_count": len(sync_records),
"push_peer_count": len(store.records_for_bucket("push")), "push_peer_count": len(push_records),
"operator_peer_count": len(operator_peers), "operator_peer_count": len(operator_peers),
"default_sync_peer_count": len(default_sync_peers), "bootstrap_seed_peer_count": len(bootstrap_seed_peers),
"default_sync_peer_count": len(bootstrap_seed_peers),
"last_bootstrap_error": bootstrap_error, "last_bootstrap_error": bootstrap_error,
} }
with _NODE_RUNTIME_LOCK: with _NODE_RUNTIME_LOCK:
@@ -1284,6 +1382,9 @@ def _peer_sync_response(peer_url: str, body: dict[str, Any]) -> dict[str, Any]:
normalized = normalize_peer_url(peer_url) normalized = normalize_peer_url(peer_url)
if not normalized: if not normalized:
raise ValueError("invalid peer URL") raise ValueError("invalid peer URL")
transport = peer_transport_kind(normalized)
if _infonet_private_transport_required() and not _is_private_infonet_transport(transport):
raise RuntimeError(_infonet_private_transport_error())
timeout = int(get_settings().MESH_RELAY_PUSH_TIMEOUT_S or 10) timeout = int(get_settings().MESH_RELAY_PUSH_TIMEOUT_S or 10)
kwargs: dict[str, Any] = { kwargs: dict[str, Any] = {
@@ -1291,7 +1392,7 @@ def _peer_sync_response(peer_url: str, body: dict[str, Any]) -> dict[str, Any]:
"timeout": timeout, "timeout": timeout,
"headers": {"Content-Type": "application/json"}, "headers": {"Content-Type": "application/json"},
} }
if peer_transport_kind(normalized) == "onion": if transport == "onion":
if not bool(get_settings().MESH_ARTI_ENABLED): if not bool(get_settings().MESH_ARTI_ENABLED):
raise RuntimeError("onion sync requires Arti to be enabled") raise RuntimeError("onion sync requires Arti to be enabled")
if not _check_arti_ready(): if not _check_arti_ready():
@@ -1406,20 +1507,37 @@ def _run_public_sync_cycle() -> SyncWorkerState:
except Exception: except Exception:
store = PeerStore(DEFAULT_PEER_STORE_PATH) store = PeerStore(DEFAULT_PEER_STORE_PATH)
peers = eligible_sync_peers(store.records(), now=time.time()) records = _filter_infonet_sync_records(store.records())
peers = eligible_sync_peers(records, now=time.time())
with _NODE_RUNTIME_LOCK: with _NODE_RUNTIME_LOCK:
current_state = get_sync_state() current_state = get_sync_state()
if not peers: if not peers:
updated = finish_solo_sync( if _infonet_private_transport_required():
current_state, updated = finish_sync(
now=time.time(), current_state,
current_head=infonet.head_hash, ok=False,
interval_s=int(get_settings().MESH_SYNC_INTERVAL_S or 300), error=_infonet_private_transport_error(),
) now=time.time(),
current_head=infonet.head_hash,
failure_backoff_s=int(get_settings().MESH_SYNC_FAILURE_BACKOFF_S or 60),
)
else:
updated = finish_solo_sync(
current_state,
now=time.time(),
current_head=infonet.head_hash,
interval_s=int(get_settings().MESH_SYNC_INTERVAL_S or 300),
)
with _NODE_RUNTIME_LOCK: with _NODE_RUNTIME_LOCK:
set_sync_state(updated) set_sync_state(updated)
return updated return updated
if _infonet_private_transport_required() and any(
str(getattr(record, "transport", "") or "").strip().lower() == "onion"
for record in peers
):
_ensure_infonet_private_transport_ready("sync")
last_error = "sync failed" last_error = "sync failed"
for record in peers: for record in peers:
started = begin_sync( started = begin_sync(
@@ -1488,6 +1606,33 @@ def _run_public_sync_cycle() -> SyncWorkerState:
) )
_NODE_SYNC_KICK_LOCK = threading.Lock()
def _kick_public_sync_background(reason: str = "") -> None:
"""Start one immediate Infonet sync attempt without waiting for the poll loop."""
if not _node_runtime_supported() or not _participant_node_enabled():
return
def _runner() -> None:
if not _NODE_SYNC_KICK_LOCK.acquire(blocking=False):
return
try:
label = f" ({reason})" if reason else ""
logger.info("Infonet sync kick starting%s", label)
_run_public_sync_cycle()
except Exception:
logger.exception("Infonet sync kick failed")
finally:
_NODE_SYNC_KICK_LOCK.release()
threading.Thread(
target=_runner,
daemon=True,
name="infonet-sync-kick",
).start()
def _public_infonet_sync_loop() -> None: def _public_infonet_sync_loop() -> None:
from services.mesh.mesh_hashchain import infonet from services.mesh.mesh_hashchain import infonet
@@ -2174,19 +2319,26 @@ async def lifespan(app: FastAPI):
# Only the primary backend supervises Wormhole. The Wormhole process itself # Only the primary backend supervises Wormhole. The Wormhole process itself
# runs this same app in MESH_ONLY mode and must not recurse into spawning. # runs this same app in MESH_ONLY mode and must not recurse into spawning.
if not _MESH_ONLY: if not _MESH_ONLY:
try: def _startup_wormhole_runtime():
from services.wormhole_supervisor import get_wormhole_state, sync_wormhole_with_settings try:
from services.wormhole_supervisor import get_wormhole_state, sync_wormhole_with_settings
sync_wormhole_with_settings() sync_wormhole_with_settings()
_resume_private_delivery_background_work( _resume_private_delivery_background_work(
current_tier=_current_private_lane_tier(get_wormhole_state()), current_tier=_current_private_lane_tier(get_wormhole_state()),
reason="startup_resume", reason="startup_resume",
) )
_refresh_lookup_handle_rotation_background(reason="startup_resume") _refresh_lookup_handle_rotation_background(reason="startup_resume")
privacy_prewarm_service.ensure_started() privacy_prewarm_service.ensure_started()
privacy_prewarm_service.run_scheduled_once(reason="startup_resume") privacy_prewarm_service.run_scheduled_once(reason="startup_resume")
except Exception as e: except Exception as e:
logger.warning(f"Wormhole supervisor failed to sync: {e}") logger.warning(f"Wormhole supervisor failed to sync: {e}")
threading.Thread(
target=_startup_wormhole_runtime,
daemon=True,
name="wormhole-startup-sync",
).start()
try: try:
from services.mesh.mesh_hashchain import register_public_event_append_hook from services.mesh.mesh_hashchain import register_public_event_append_hook
@@ -2194,9 +2346,16 @@ async def lifespan(app: FastAPI):
_refresh_node_peer_store() _refresh_node_peer_store()
if _node_runtime_supported(): if _node_runtime_supported():
if not _participant_node_enabled(): if not _participant_node_enabled():
set_sync_state(_set_node_sync_disabled_state()) logger.info("Infonet participant auto-enabled for private seed sync")
_set_participant_node_enabled(True)
threading.Thread(
target=lambda: _ensure_infonet_private_transport_ready("startup"),
daemon=True,
name="infonet-private-transport-warmup",
).start()
_NODE_SYNC_STOP.clear() _NODE_SYNC_STOP.clear()
threading.Thread(target=_public_infonet_sync_loop, daemon=True).start() threading.Thread(target=_public_infonet_sync_loop, daemon=True).start()
_kick_public_sync_background("startup")
threading.Thread(target=_http_peer_push_loop, daemon=True).start() threading.Thread(target=_http_peer_push_loop, daemon=True).start()
threading.Thread(target=_http_gate_push_loop, daemon=True).start() threading.Thread(target=_http_gate_push_loop, daemon=True).start()
threading.Thread(target=_http_gate_pull_loop, daemon=True).start() threading.Thread(target=_http_gate_pull_loop, daemon=True).start()
@@ -2232,6 +2391,11 @@ async def lifespan(app: FastAPI):
threading.Thread(target=_prime_aircraft_database, daemon=True).start() threading.Thread(target=_prime_aircraft_database, daemon=True).start()
# Seed cached first-paint layers before accepting requests. This is
# disk-only and keeps the critical bootstrap endpoint independent from
# slow network warmup.
seed_startup_caches()
# Start the recurring scheduler (fast=60s, slow=30min). # Start the recurring scheduler (fast=60s, slow=30min).
start_scheduler() start_scheduler()
@@ -2239,6 +2403,9 @@ async def lifespan(app: FastAPI):
# is listening on port 8000 instantly. The frontend's adaptive polling # is listening on port 8000 instantly. The frontend's adaptive polling
# (retries every 3s) will pick up data piecemeal as each fetcher finishes. # (retries every 3s) will pick up data piecemeal as each fetcher finishes.
def _background_preload(): def _background_preload():
delay_s = float(os.environ.get("SHADOWBROKER_STARTUP_PRELOAD_DELAY_S", "2.0") or 0)
if delay_s > 0:
time.sleep(delay_s)
logger.info("=== PRELOADING DATA (background — server already accepting requests) ===") logger.info("=== PRELOADING DATA (background — server already accepting requests) ===")
try: try:
update_all_data(startup_mode=True) update_all_data(startup_mode=True)
@@ -3472,6 +3639,46 @@ def _sigint_totals_for_items(items: list) -> dict[str, int]:
return totals return totals
def _cap_startup_items(items: list | None, max_items: int) -> list:
if not items:
return []
if len(items) <= max_items:
return items
return items[:max_items]
def _cap_fast_startup_payload(payload: dict) -> dict:
"""Trim high-volume layers for the first dashboard paint.
The full fast payload can legitimately contain tens of thousands of AIS,
ADS-B, SIGINT, and CCTV records. Returning all of that during app startup
blocks the first map render behind serialization/proxy/network pressure.
This startup payload paints representative live data immediately; the next
normal poll replaces it with the full dataset.
"""
capped = dict(payload)
capped["commercial_flights"] = _cap_startup_items(capped.get("commercial_flights"), 800)
capped["private_flights"] = _cap_startup_items(capped.get("private_flights"), 300)
capped["private_jets"] = _cap_startup_items(capped.get("private_jets"), 150)
capped["ships"] = _cap_startup_items(capped.get("ships"), 1500)
capped["cctv"] = []
capped["sigint"] = _cap_startup_items(capped.get("sigint"), 500)
capped["trains"] = _cap_startup_items(capped.get("trains"), 100)
capped["startup_payload"] = True
return capped
def _cap_fast_dashboard_payload(payload: dict) -> dict:
capped = dict(payload)
capped["commercial_flights"] = _downsample_points(capped.get("commercial_flights") or [], 6000)
capped["private_flights"] = _downsample_points(capped.get("private_flights") or [], 1500)
capped["private_jets"] = _downsample_points(capped.get("private_jets") or [], 1500)
capped["ships"] = _downsample_points(capped.get("ships") or [], 8000)
capped["cctv"] = _downsample_points(capped.get("cctv") or [], 2500)
capped["sigint"] = _downsample_points(capped.get("sigint") or [], 5000)
return capped
@app.get("/api/live-data/fast") @app.get("/api/live-data/fast")
@limiter.limit("120/minute") @limiter.limit("120/minute")
async def live_data_fast( async def live_data_fast(
@@ -3482,8 +3689,9 @@ async def live_data_fast(
w: float = Query(None, description="West bound (ignored)", ge=-180, le=180), w: float = Query(None, description="West bound (ignored)", ge=-180, le=180),
n: float = Query(None, description="North bound (ignored)", ge=-90, le=90), n: float = Query(None, description="North bound (ignored)", ge=-90, le=90),
e: float = Query(None, description="East bound (ignored)", ge=-180, le=180), e: float = Query(None, description="East bound (ignored)", ge=-180, le=180),
initial: bool = Query(False, description="Return a capped startup payload for first paint"),
): ):
etag = _current_etag(prefix="fast|full|") etag = _current_etag(prefix="fast|initial|" if initial else "fast|full|")
if request.headers.get("if-none-match") == etag: if request.headers.get("if-none-match") == etag:
return Response(status_code=304, headers={"ETag": etag, "Cache-Control": "no-cache"}) return Response(status_code=304, headers={"ETag": etag, "Cache-Control": "no-cache"})
@@ -3548,6 +3756,10 @@ async def live_data_fast(
"trains": (d.get("trains") or []) if active_layers.get("trains", True) else [], "trains": (d.get("trains") or []) if active_layers.get("trains", True) else [],
"freshness": freshness, "freshness": freshness,
} }
if initial:
payload = _cap_fast_startup_payload(payload)
else:
payload = _cap_fast_dashboard_payload(payload)
return Response( return Response(
content=orjson.dumps(_sanitize_payload(payload)), content=orjson.dumps(_sanitize_payload(payload)),
media_type="application/json", media_type="application/json",
@@ -3609,6 +3821,7 @@ async def live_data_slow(
"correlations", "correlations",
"threat_level", "threat_level",
"trending_markets", "trending_markets",
"fimi",
"uap_sightings", "uap_sightings",
"wastewater", "wastewater",
"sar_scenes", "sar_scenes",
@@ -3621,6 +3834,7 @@ async def live_data_slow(
"last_updated": d.get("last_updated"), "last_updated": d.get("last_updated"),
"threat_level": d.get("threat_level"), "threat_level": d.get("threat_level"),
"trending_markets": d.get("trending_markets", []), "trending_markets": d.get("trending_markets", []),
"fimi": d.get("fimi", {}),
"news": d.get("news", []), "news": d.get("news", []),
"stocks": d.get("stocks", {}), "stocks": d.get("stocks", {}),
"financial_source": d.get("financial_source", ""), "financial_source": d.get("financial_source", ""),
@@ -7604,6 +7818,13 @@ _CCTV_PROXY_ALLOWED_HOSTS = {
"infocar.dgt.es", # Spain DGT "infocar.dgt.es", # Spain DGT
"informo.madrid.es", # Madrid "informo.madrid.es", # Madrid
"www.windy.com", "www.windy.com",
"imgproxy.windy.com", # Windy preview image CDN
"www.lakecountypassage.com", # Illinois Lake County PASSAGE snapshots
"webcam.forkswa.com", # WSDOT partner public camera
"webcam.sunmountainlodge.com", # WSDOT partner public camera
"www.nps.gov", # WSDOT-linked Mount Rainier camera
"home.lewiscounty.com", # WSDOT partner public camera
"www.seattle.gov", # Seattle traffic camera media linked from WSDOT
} }
@@ -7729,6 +7950,16 @@ def _cctv_proxy_profile_for_url(target_url: str) -> _CCTVProxyProfile:
cache_seconds=30, cache_seconds=30,
headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8"}, headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8"},
) )
if host == "www.lakecountypassage.com":
return _CCTVProxyProfile(
name="lake-county-passage",
timeout=(5.0, 12.0),
cache_seconds=30,
headers={
"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8",
"Referer": "https://www.lakecountypassage.com/",
},
)
if host in {"mdotjboss.state.mi.us", "micamerasimages.net"}: if host in {"mdotjboss.state.mi.us", "micamerasimages.net"}:
return _CCTVProxyProfile( return _CCTVProxyProfile(
name="michigan-dot", name="michigan-dot",
@@ -7791,11 +8022,27 @@ def _cctv_proxy_profile_for_url(target_url: str) -> _CCTVProxyProfile:
"Referer": "https://informo.madrid.es/", "Referer": "https://informo.madrid.es/",
}, },
) )
if host == "www.windy.com": if host in {"www.windy.com", "imgproxy.windy.com"}:
return _CCTVProxyProfile( return _CCTVProxyProfile(
name="windy-webcams", name="windy-webcams",
timeout=(5.0, 12.0), timeout=(5.0, 12.0),
cache_seconds=60, cache_seconds=60,
headers={
"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8",
"Referer": "https://www.windy.com/",
},
)
if host in {
"webcam.forkswa.com",
"webcam.sunmountainlodge.com",
"www.nps.gov",
"home.lewiscounty.com",
"www.seattle.gov",
}:
return _CCTVProxyProfile(
name="wsdot-partner",
timeout=(5.0, 12.0),
cache_seconds=30,
headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8"}, headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8"},
) )
return _CCTVProxyProfile( return _CCTVProxyProfile(
@@ -7839,6 +8086,30 @@ def _cctv_response_headers(resp, cache_seconds: int, include_length: bool = True
return headers return headers
def _infer_cctv_media_type_from_url(target_url: str, content_type: str) -> str:
from urllib.parse import urlparse
normalized_type = str(content_type or "").split(";", 1)[0].strip().lower()
if normalized_type and normalized_type not in {"application/octet-stream", "binary/octet-stream"}:
return content_type
path = str(urlparse(target_url).path or "").lower()
if path.endswith((".jpg", ".jpeg")):
return "image/jpeg"
if path.endswith(".png"):
return "image/png"
if path.endswith(".gif"):
return "image/gif"
if path.endswith(".webp"):
return "image/webp"
if path.endswith(".mp4"):
return "video/mp4"
if path.endswith(".webm"):
return "video/webm"
if path.endswith(".m3u8"):
return "application/vnd.apple.mpegurl"
return content_type or "application/octet-stream"
def _fetch_cctv_upstream_response(request: Request, target_url: str, profile: _CCTVProxyProfile): def _fetch_cctv_upstream_response(request: Request, target_url: str, profile: _CCTVProxyProfile):
import requests as _req import requests as _req
@@ -7872,7 +8143,10 @@ def _proxy_cctv_media_response(request: Request, target_url: str):
profile = _cctv_proxy_profile_for_url(target_url) profile = _cctv_proxy_profile_for_url(target_url)
resp = _fetch_cctv_upstream_response(request, target_url, profile) resp = _fetch_cctv_upstream_response(request, target_url, profile)
content_type = resp.headers.get("Content-Type", "application/octet-stream") content_type = _infer_cctv_media_type_from_url(
target_url,
resp.headers.get("Content-Type", "application/octet-stream"),
)
is_hls_playlist = ( is_hls_playlist = (
".m3u8" in str(parsed.path or "").lower() ".m3u8" in str(parsed.path or "").lower()
or "mpegurl" in content_type.lower() or "mpegurl" in content_type.lower()
@@ -8661,7 +8935,10 @@ async def api_get_node_settings(request: Request):
@limiter.limit("10/minute") @limiter.limit("10/minute")
async def api_set_node_settings(request: Request, body: NodeSettingsUpdate): async def api_set_node_settings(request: Request, body: NodeSettingsUpdate):
_refresh_node_peer_store() _refresh_node_peer_store()
return _set_participant_node_enabled(bool(body.enabled)) result = _set_participant_node_enabled(bool(body.enabled))
if bool(body.enabled):
_kick_public_sync_background("operator_enable")
return result
@app.get("/api/settings/wormhole") @app.get("/api/settings/wormhole")
@@ -9388,18 +9665,29 @@ async def api_wormhole_join(request: Request):
existing = read_wormhole_settings() existing = read_wormhole_settings()
updated = write_wormhole_settings( updated = write_wormhole_settings(
enabled=True, enabled=True,
transport="direct", transport="tor_arti",
socks_proxy="", socks_proxy=f"socks5h://127.0.0.1:{int(get_settings().MESH_ARTI_SOCKS_PORT or 9050)}",
socks_dns=True, socks_dns=True,
anonymous_mode=False, anonymous_mode=True,
) )
transport_changed = ( transport_changed = (
str(existing.get("transport", "direct")) != "direct" str(existing.get("transport", "direct")) != "tor_arti"
or str(existing.get("socks_proxy", "")) != "" or str(existing.get("socks_proxy", "")) != str(updated.get("socks_proxy", ""))
or bool(existing.get("socks_dns", True)) is not True or bool(existing.get("socks_dns", True)) is not True
or bool(existing.get("anonymous_mode", False)) is not False or bool(existing.get("anonymous_mode", False)) is not True
or bool(existing.get("enabled", False)) is not True or bool(existing.get("enabled", False)) is not True
) )
tor_result: dict[str, Any] = {"ok": False, "detail": "not started"}
try:
from services.tor_hidden_service import tor_service
from routers.ai_intel import _write_env_value
tor_result = await asyncio.to_thread(tor_service.start)
if tor_result.get("ok"):
_write_env_value("MESH_ARTI_ENABLED", "true")
get_settings.cache_clear()
except Exception as exc:
tor_result = {"ok": False, "detail": str(exc or type(exc).__name__)}
bootstrap_wormhole_identity() bootstrap_wormhole_identity()
bootstrap_wormhole_persona_state() bootstrap_wormhole_persona_state()
state = ( state = (
@@ -9421,6 +9709,7 @@ async def api_wormhole_join(request: Request):
"identity": get_transport_identity(), "identity": get_transport_identity(),
"runtime": state, "runtime": state,
"settings": updated, "settings": updated,
"tor": tor_result,
} }
@@ -9430,10 +9719,9 @@ async def api_wormhole_leave(request: Request):
updated = write_wormhole_settings(enabled=False) updated = write_wormhole_settings(enabled=False)
state = disconnect_wormhole(reason="leave_wormhole") state = disconnect_wormhole(reason="leave_wormhole")
# Disable node participation when the user leaves the Wormhole. # Leaving private DM mode must not disable Infonet participation. Infonet
from services.node_settings import write_node_settings # sync has its own private transport warmup and can remain connected to
# seed/peer nodes while MeshChat stays separately opt-in.
write_node_settings(enabled=False)
return { return {
"ok": True, "ok": True,
+61 -14
View File
@@ -96,9 +96,10 @@ def _participant_node_enabled() -> bool:
def _node_runtime_snapshot() -> dict[str, Any]: def _node_runtime_snapshot() -> dict[str, Any]:
with _NODE_RUNTIME_LOCK: with _NODE_RUNTIME_LOCK:
return { return {
"node_mode": _NODE_BOOTSTRAP_STATE.get("node_mode", "participant"), "node_mode": _current_node_mode(),
"node_enabled": _participant_node_enabled(), "node_enabled": _participant_node_enabled(),
"bootstrap": dict(_NODE_BOOTSTRAP_STATE), "private_transport_required": _infonet_private_transport_required(),
"bootstrap": {**dict(_NODE_BOOTSTRAP_STATE), "node_mode": _current_node_mode()},
"sync_runtime": get_sync_state().to_dict(), "sync_runtime": get_sync_state().to_dict(),
"push_runtime": dict(_NODE_PUSH_STATE), "push_runtime": dict(_NODE_PUSH_STATE),
} }
@@ -131,6 +132,30 @@ def _set_participant_node_enabled(enabled: bool) -> dict[str, Any]:
} }
def _infonet_private_transport_required() -> bool:
from services.config import get_settings
return not bool(getattr(get_settings(), "MESH_INFONET_ALLOW_CLEARNET_SYNC", False))
def _infonet_private_transport_error() -> str:
return "private Infonet requires onion/RNS transport; no clearnet sync fallback"
def _is_private_infonet_transport(transport: str) -> bool:
return str(transport or "").strip().lower() in {"onion", "rns"}
def _configured_bootstrap_seed_peer_urls() -> list[str]:
from services.config import get_settings
from services.mesh.mesh_router import parse_configured_relay_peers
settings = get_settings()
primary = str(getattr(settings, "MESH_BOOTSTRAP_SEED_PEERS", "") or "").strip()
legacy = str(getattr(settings, "MESH_DEFAULT_SYNC_PEERS", "") or "").strip()
return parse_configured_relay_peers(primary or legacy)
def _refresh_node_peer_store(*, now: float | None = None) -> dict[str, Any]: def _refresh_node_peer_store(*, now: float | None = None) -> dict[str, Any]:
from services.config import get_settings from services.config import get_settings
from services.mesh.mesh_bootstrap_manifest import load_bootstrap_manifest_from_settings from services.mesh.mesh_bootstrap_manifest import load_bootstrap_manifest_from_settings
@@ -155,14 +180,17 @@ def _refresh_node_peer_store(*, now: float | None = None) -> dict[str, Any]:
except Exception: except Exception:
store = PeerStore(DEFAULT_PEER_STORE_PATH) store = PeerStore(DEFAULT_PEER_STORE_PATH)
private_transport_required = _infonet_private_transport_required()
operator_peers = configured_relay_peer_urls() operator_peers = configured_relay_peer_urls()
default_sync_peers = parse_configured_relay_peers( bootstrap_seed_peers = _configured_bootstrap_seed_peer_urls()
str(getattr(get_settings(), "MESH_DEFAULT_SYNC_PEERS", "") or "") skipped_clearnet_peers = 0
)
for peer_url in operator_peers: for peer_url in operator_peers:
transport = peer_transport_kind(peer_url) transport = peer_transport_kind(peer_url)
if not transport: if not transport:
continue continue
if private_transport_required and not _is_private_infonet_transport(transport):
skipped_clearnet_peers += 1
continue
store.upsert( store.upsert(
make_sync_peer_record( make_sync_peer_record(
peer_url=peer_url, peer_url=peer_url,
@@ -183,19 +211,22 @@ def _refresh_node_peer_store(*, now: float | None = None) -> dict[str, Any]:
) )
operator_peer_set = set(operator_peers) operator_peer_set = set(operator_peers)
for peer_url in default_sync_peers: for peer_url in bootstrap_seed_peers:
if peer_url in operator_peer_set: if peer_url in operator_peer_set:
continue continue
transport = peer_transport_kind(peer_url) transport = peer_transport_kind(peer_url)
if not transport: if not transport:
continue continue
if private_transport_required and not _is_private_infonet_transport(transport):
skipped_clearnet_peers += 1
continue
store.upsert( store.upsert(
make_bootstrap_peer_record( make_bootstrap_peer_record(
peer_url=peer_url, peer_url=peer_url,
transport=transport, transport=transport,
role="seed", role="seed",
label="ShadowBroker default seed", label="ShadowBroker bootstrap seed",
signer_id="shadowbroker-default", signer_id="shadowbroker-bootstrap",
now=timestamp, now=timestamp,
) )
) )
@@ -205,8 +236,8 @@ def _refresh_node_peer_store(*, now: float | None = None) -> dict[str, Any]:
transport=transport, transport=transport,
role="seed", role="seed",
source="bundle", source="bundle",
label="ShadowBroker default seed", label="ShadowBroker bootstrap seed",
signer_id="shadowbroker-default", signer_id="shadowbroker-bootstrap",
now=timestamp, now=timestamp,
) )
) )
@@ -220,6 +251,9 @@ def _refresh_node_peer_store(*, now: float | None = None) -> dict[str, Any]:
if manifest is not None: if manifest is not None:
for peer in manifest.peers: for peer in manifest.peers:
if private_transport_required and not _is_private_infonet_transport(peer.transport):
skipped_clearnet_peers += 1
continue
store.upsert( store.upsert(
make_bootstrap_peer_record( make_bootstrap_peer_record(
peer_url=peer.peer_url, peer_url=peer.peer_url,
@@ -242,17 +276,30 @@ def _refresh_node_peer_store(*, now: float | None = None) -> dict[str, Any]:
) )
) )
if private_transport_required and skipped_clearnet_peers and not bootstrap_error:
bootstrap_error = _infonet_private_transport_error()
store.save() store.save()
bootstrap_records = store.records_for_bucket("bootstrap")
sync_records = store.records_for_bucket("sync")
push_records = store.records_for_bucket("push")
if private_transport_required:
bootstrap_records = [record for record in bootstrap_records if _is_private_infonet_transport(record.transport)]
sync_records = [record for record in sync_records if _is_private_infonet_transport(record.transport)]
push_records = [record for record in push_records if _is_private_infonet_transport(record.transport)]
snapshot = { snapshot = {
"node_mode": mode, "node_mode": mode,
"private_transport_required": private_transport_required,
"skipped_clearnet_peer_count": skipped_clearnet_peers,
"manifest_loaded": manifest is not None, "manifest_loaded": manifest is not None,
"manifest_signer_id": manifest.signer_id if manifest is not None else "", "manifest_signer_id": manifest.signer_id if manifest is not None else "",
"manifest_valid_until": int(manifest.valid_until or 0) if manifest is not None else 0, "manifest_valid_until": int(manifest.valid_until or 0) if manifest is not None else 0,
"bootstrap_peer_count": len(store.records_for_bucket("bootstrap")), "bootstrap_peer_count": len(bootstrap_records),
"sync_peer_count": len(store.records_for_bucket("sync")), "sync_peer_count": len(sync_records),
"push_peer_count": len(store.records_for_bucket("push")), "push_peer_count": len(push_records),
"operator_peer_count": len(operator_peers), "operator_peer_count": len(operator_peers),
"default_sync_peer_count": len(default_sync_peers), "bootstrap_seed_peer_count": len(bootstrap_seed_peers),
"default_sync_peer_count": len(bootstrap_seed_peers),
"last_bootstrap_error": bootstrap_error, "last_bootstrap_error": bootstrap_error,
} }
with _NODE_RUNTIME_LOCK: with _NODE_RUNTIME_LOCK:
+7 -6
View File
@@ -7,7 +7,7 @@ py-modules = []
[project] [project]
name = "backend" name = "backend"
version = "0.9.7" version = "0.9.75"
requires-python = ">=3.10" requires-python = ">=3.10"
dependencies = [ dependencies = [
"apscheduler==3.10.3", "apscheduler==3.10.3",
@@ -18,15 +18,16 @@ dependencies = [
"fastapi==0.115.12", "fastapi==0.115.12",
"feedparser==6.0.10", "feedparser==6.0.10",
"httpx==0.28.1", "httpx==0.28.1",
"playwright==1.50.0", "playwright==1.59.0",
"playwright-stealth==1.0.6", "playwright-stealth==1.0.6",
"pydantic==2.11.1", "pydantic==2.13.3",
"pydantic-settings==2.8.1", "pydantic-settings==2.8.1",
"pystac-client==0.8.6", "pystac-client==0.8.6",
"python-dotenv==1.2.2", "python-dotenv==1.2.2",
"requests==2.31.0", "requests==2.31.0",
"PySocks==1.7.1",
"reverse-geocoder==1.5.1", "reverse-geocoder==1.5.1",
"sgp4==2.23", "sgp4==2.25",
"meshtastic>=2.5.0", "meshtastic>=2.5.0",
"orjson>=3.10.0", "orjson>=3.10.0",
"paho-mqtt>=1.6.0,<2.0.0", "paho-mqtt>=1.6.0,<2.0.0",
@@ -34,7 +35,7 @@ dependencies = [
"slowapi==0.1.9", "slowapi==0.1.9",
"vaderSentiment>=3.3.0", "vaderSentiment>=3.3.0",
"uvicorn==0.34.0", "uvicorn==0.34.0",
"yfinance==0.2.54", "yfinance==1.3.0",
] ]
[dependency-groups] [dependency-groups]
@@ -42,7 +43,7 @@ dev = ["pytest>=8.3.4", "pytest-asyncio==0.25.0", "ruff>=0.9.0", "black>=24.0.0"
[tool.ruff.lint] [tool.ruff.lint]
# The current backend carries historical style debt in large legacy modules. # The current backend carries historical style debt in large legacy modules.
# Keep CI focused on actionable correctness checks for the v0.9.7 release. # Keep CI focused on actionable correctness checks for the v0.9.75 release.
ignore = ["E401", "E402", "E701", "E731", "E741", "F401", "F402", "F541", "F811", "F841"] ignore = ["E401", "E402", "E701", "E731", "E741", "F401", "F402", "F541", "F811", "F841"]
[tool.black] [tool.black]
+98 -2
View File
@@ -28,13 +28,46 @@ class TimeMachineToggle(BaseModel):
enabled: bool enabled: bool
@router.get("/api/settings/api-keys", dependencies=[Depends(require_admin)]) class MeshtasticMqttUpdate(BaseModel):
enabled: bool | None = None
broker: str | None = None
port: int | None = None
username: str | None = None
password: str | None = None
psk: str | None = None
include_default_roots: bool | None = None
extra_roots: str | None = None
extra_topics: str | None = None
@router.get("/api/settings/api-keys", dependencies=[Depends(require_local_operator)])
@limiter.limit("30/minute") @limiter.limit("30/minute")
async def api_get_keys(request: Request): async def api_get_keys(request: Request):
from services.api_settings import get_api_keys from services.api_settings import get_api_keys
return get_api_keys() return get_api_keys()
@router.put("/api/settings/api-keys", dependencies=[Depends(require_local_operator)])
@limiter.limit("10/minute")
async def api_save_keys(request: Request):
from services.api_settings import save_api_keys
body = await request.json()
if not isinstance(body, dict):
return Response(
content=json_mod.dumps({"ok": False, "detail": "Expected a JSON object."}),
status_code=400,
media_type="application/json",
)
result = save_api_keys({str(k): str(v) for k, v in body.items()})
if result.get("ok"):
return result
return Response(
content=json_mod.dumps(result),
status_code=400,
media_type="application/json",
)
@router.get("/api/settings/api-keys/meta") @router.get("/api/settings/api-keys/meta")
@limiter.limit("30/minute") @limiter.limit("30/minute")
async def api_get_keys_meta(request: Request): async def api_get_keys_meta(request: Request):
@@ -99,7 +132,70 @@ async def api_get_node_settings(request: Request):
@limiter.limit("10/minute") @limiter.limit("10/minute")
async def api_set_node_settings(request: Request, body: NodeSettingsUpdate): async def api_set_node_settings(request: Request, body: NodeSettingsUpdate):
_refresh_node_peer_store() _refresh_node_peer_store()
return _set_participant_node_enabled(bool(body.enabled)) result = _set_participant_node_enabled(bool(body.enabled))
if bool(body.enabled):
try:
import main as _main
_main._kick_public_sync_background("operator_enable")
except Exception:
logger.debug("Unable to kick Infonet sync after node enable", exc_info=True)
return result
def _meshtastic_runtime_snapshot() -> dict[str, Any]:
from services.meshtastic_mqtt_settings import redacted_meshtastic_mqtt_settings
from services.sigint_bridge import sigint_grid
return {
**redacted_meshtastic_mqtt_settings(),
"runtime": sigint_grid.mesh.status(),
}
@router.get("/api/settings/meshtastic-mqtt", dependencies=[Depends(require_local_operator)])
@limiter.limit("30/minute")
async def api_get_meshtastic_mqtt_settings(request: Request):
return _meshtastic_runtime_snapshot()
@router.put("/api/settings/meshtastic-mqtt", dependencies=[Depends(require_local_operator)])
@limiter.limit("10/minute")
async def api_set_meshtastic_mqtt_settings(request: Request, body: MeshtasticMqttUpdate):
from services.meshtastic_mqtt_settings import write_meshtastic_mqtt_settings
from services.sigint_bridge import sigint_grid
updates = body.model_dump(exclude_unset=True)
# Empty secret fields mean "keep existing"; explicit non-empty values replace.
if updates.get("password") == "":
updates.pop("password", None)
if updates.get("psk") == "":
updates.pop("psk", None)
enabled_requested = updates.get("enabled")
settings = write_meshtastic_mqtt_settings(**updates)
if enabled_requested is True:
# Public MQTT and Wormhole are intentionally mutually exclusive lanes.
try:
from services.wormhole_settings import write_wormhole_settings
from services.wormhole_supervisor import disconnect_wormhole
write_wormhole_settings(enabled=False)
disconnect_wormhole(reason="public_mesh_enabled")
except Exception as exc:
logger.warning("Failed to disable Wormhole while enabling public mesh: %s", exc)
if bool(settings.get("enabled")):
if sigint_grid.mesh.is_running():
sigint_grid.mesh.stop()
threading.Timer(1.0, sigint_grid.mesh.start).start()
else:
sigint_grid.mesh.start()
else:
sigint_grid.mesh.stop()
return _meshtastic_runtime_snapshot()
@router.get("/api/settings/timemachine") @router.get("/api/settings/timemachine")
+2 -2
View File
@@ -1585,7 +1585,7 @@ async def agent_tool_manifest(request: Request):
return { return {
"ok": True, "ok": True,
"version": "0.9.7", "version": "0.9.75",
"access_tier": access_tier, "access_tier": access_tier,
"available_commands": available_commands, "available_commands": available_commands,
"transport": { "transport": {
@@ -2221,7 +2221,7 @@ async def api_capabilities(request: Request):
access_tier = str(get_settings().OPENCLAW_ACCESS_TIER or "restricted").strip().lower() access_tier = str(get_settings().OPENCLAW_ACCESS_TIER or "restricted").strip().lower()
return { return {
"ok": True, "ok": True,
"version": "0.9.7", "version": "0.9.75",
"auth": { "auth": {
"method": "HMAC-SHA256", "method": "HMAC-SHA256",
"headers": ["X-SB-Timestamp", "X-SB-Nonce", "X-SB-Signature"], "headers": ["X-SB-Timestamp", "X-SB-Nonce", "X-SB-Signature"],
+67 -22
View File
@@ -11,6 +11,8 @@ logger = logging.getLogger(__name__)
router = APIRouter() router = APIRouter()
_CCTV_PROXY_CONNECT_TIMEOUT_S = 2.0
_CCTV_PROXY_ALLOWED_HOSTS = { _CCTV_PROXY_ALLOWED_HOSTS = {
"s3-eu-west-1.amazonaws.com", "s3-eu-west-1.amazonaws.com",
"jamcams.tfl.gov.uk", "jamcams.tfl.gov.uk",
@@ -46,13 +48,20 @@ _CCTV_PROXY_ALLOWED_HOSTS = {
"infocar.dgt.es", "infocar.dgt.es",
"informo.madrid.es", "informo.madrid.es",
"www.windy.com", "www.windy.com",
"imgproxy.windy.com",
"www.lakecountypassage.com",
"webcam.forkswa.com",
"webcam.sunmountainlodge.com",
"www.nps.gov",
"home.lewiscounty.com",
"www.seattle.gov",
} }
@dataclass(frozen=True) @dataclass(frozen=True)
class _CCTVProxyProfile: class _CCTVProxyProfile:
name: str name: str
timeout: tuple = (5.0, 10.0) timeout: tuple = (_CCTV_PROXY_CONNECT_TIMEOUT_S, 8.0)
cache_seconds: int = 30 cache_seconds: int = 30
headers: dict = field(default_factory=dict) headers: dict = field(default_factory=dict)
@@ -80,69 +89,78 @@ def _cctv_proxy_profile_for_url(target_url: str) -> _CCTVProxyProfile:
path = str(parsed.path or "").strip().lower() path = str(parsed.path or "").strip().lower()
if host in {"jamcams.tfl.gov.uk", "s3-eu-west-1.amazonaws.com"}: if host in {"jamcams.tfl.gov.uk", "s3-eu-west-1.amazonaws.com"}:
return _CCTVProxyProfile(name="tfl-jamcam", timeout=(5.0, 20.0), cache_seconds=15, return _CCTVProxyProfile(name="tfl-jamcam", timeout=(_CCTV_PROXY_CONNECT_TIMEOUT_S, 20.0), cache_seconds=15,
headers={"Accept": "video/mp4,image/avif,image/webp,image/apng,image/*,*/*;q=0.8", "Referer": "https://tfl.gov.uk/"}) headers={"Accept": "video/mp4,image/avif,image/webp,image/apng,image/*,*/*;q=0.8", "Referer": "https://tfl.gov.uk/"})
if host == "images.data.gov.sg": if host == "images.data.gov.sg":
return _CCTVProxyProfile(name="lta-singapore", timeout=(5.0, 10.0), cache_seconds=30, return _CCTVProxyProfile(name="lta-singapore", timeout=(_CCTV_PROXY_CONNECT_TIMEOUT_S, 10.0), cache_seconds=30,
headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8"}) headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8"})
if host == "cctv.austinmobility.io": if host == "cctv.austinmobility.io":
return _CCTVProxyProfile(name="austin-mobility", timeout=(5.0, 8.0), cache_seconds=15, return _CCTVProxyProfile(name="austin-mobility", timeout=(_CCTV_PROXY_CONNECT_TIMEOUT_S, 8.0), cache_seconds=15,
headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8", headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8",
"Referer": "https://data.mobility.austin.gov/", "Origin": "https://data.mobility.austin.gov"}) "Referer": "https://data.mobility.austin.gov/", "Origin": "https://data.mobility.austin.gov"})
if host == "webcams.nyctmc.org": if host == "webcams.nyctmc.org":
return _CCTVProxyProfile(name="nyc-dot", timeout=(5.0, 10.0), cache_seconds=15, return _CCTVProxyProfile(name="nyc-dot", timeout=(_CCTV_PROXY_CONNECT_TIMEOUT_S, 10.0), cache_seconds=15,
headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8"}) headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8"})
if host in {"cwwp2.dot.ca.gov", "wzmedia.dot.ca.gov"}: if host in {"cwwp2.dot.ca.gov", "wzmedia.dot.ca.gov"}:
return _CCTVProxyProfile(name="caltrans", timeout=(5.0, 15.0), cache_seconds=15, return _CCTVProxyProfile(name="caltrans", timeout=(_CCTV_PROXY_CONNECT_TIMEOUT_S, 15.0), cache_seconds=15,
headers={"Accept": "application/vnd.apple.mpegurl,application/x-mpegURL,video/*,image/*,*/*;q=0.8", headers={"Accept": "application/vnd.apple.mpegurl,application/x-mpegURL,video/*,image/*,*/*;q=0.8",
"Referer": "https://cwwp2.dot.ca.gov/"}) "Referer": "https://cwwp2.dot.ca.gov/"})
if host in {"images.wsdot.wa.gov", "olypen.com", "flyykm.com", "cam.pangbornairport.com"}: if host in {"images.wsdot.wa.gov", "olypen.com", "flyykm.com", "cam.pangbornairport.com"}:
return _CCTVProxyProfile(name="wsdot", timeout=(5.0, 12.0), cache_seconds=30, return _CCTVProxyProfile(name="wsdot", timeout=(_CCTV_PROXY_CONNECT_TIMEOUT_S, 12.0), cache_seconds=30,
headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8"}) headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8"})
if host in {"www.lakecountypassage.com", "webcam.forkswa.com", "webcam.sunmountainlodge.com", "home.lewiscounty.com", "www.seattle.gov"}:
return _CCTVProxyProfile(name="regional-cctv-image", timeout=(_CCTV_PROXY_CONNECT_TIMEOUT_S, 10.0), cache_seconds=45,
headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8",
"Referer": f"https://{host}/"})
if host == "www.nps.gov":
return _CCTVProxyProfile(name="nps-webcam", timeout=(_CCTV_PROXY_CONNECT_TIMEOUT_S, 10.0), cache_seconds=60,
headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8",
"Referer": "https://www.nps.gov/"})
if host in {"navigator-c2c.dot.ga.gov", "navigator-c2c.ga.gov", "navigator-csc.dot.ga.gov"}: if host in {"navigator-c2c.dot.ga.gov", "navigator-c2c.ga.gov", "navigator-csc.dot.ga.gov"}:
read_timeout = 18.0 if "/snapshots/" in path else 12.0 read_timeout = 18.0 if "/snapshots/" in path else 12.0
return _CCTVProxyProfile(name="gdot-snapshot", timeout=(5.0, read_timeout), cache_seconds=15, return _CCTVProxyProfile(name="gdot-snapshot", timeout=(_CCTV_PROXY_CONNECT_TIMEOUT_S, read_timeout), cache_seconds=15,
headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8", headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8",
"Referer": "http://navigator-c2c.dot.ga.gov/"}) "Referer": "http://navigator-c2c.dot.ga.gov/"})
if host == "511ga.org": if host == "511ga.org":
return _CCTVProxyProfile(name="gdot-511ga-image", timeout=(5.0, 12.0), cache_seconds=15, return _CCTVProxyProfile(name="gdot-511ga-image", timeout=(_CCTV_PROXY_CONNECT_TIMEOUT_S, 12.0), cache_seconds=15,
headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8", headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8",
"Referer": "https://511ga.org/cctv"}) "Referer": "https://511ga.org/cctv"})
if host.startswith("vss") and host.endswith("dot.ga.gov"): if host.startswith("vss") and host.endswith("dot.ga.gov"):
return _CCTVProxyProfile(name="gdot-hls", timeout=(5.0, 20.0), cache_seconds=10, return _CCTVProxyProfile(name="gdot-hls", timeout=(_CCTV_PROXY_CONNECT_TIMEOUT_S, 20.0), cache_seconds=10,
headers={"Accept": "application/vnd.apple.mpegurl,application/x-mpegURL,video/*,*/*;q=0.8", headers={"Accept": "application/vnd.apple.mpegurl,application/x-mpegURL,video/*,*/*;q=0.8",
"Referer": "http://navigator-c2c.dot.ga.gov/"}) "Referer": "http://navigator-c2c.dot.ga.gov/"})
if host in {"gettingaroundillinois.com", "cctv.travelmidwest.com"}: if host in {"gettingaroundillinois.com", "cctv.travelmidwest.com"}:
return _CCTVProxyProfile(name="illinois-dot", timeout=(5.0, 12.0), cache_seconds=30, return _CCTVProxyProfile(name="illinois-dot", timeout=(_CCTV_PROXY_CONNECT_TIMEOUT_S, 12.0), cache_seconds=30,
headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8"}) headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8"})
if host in {"mdotjboss.state.mi.us", "micamerasimages.net"}: if host in {"mdotjboss.state.mi.us", "micamerasimages.net"}:
return _CCTVProxyProfile(name="michigan-dot", timeout=(5.0, 12.0), cache_seconds=30, return _CCTVProxyProfile(name="michigan-dot", timeout=(_CCTV_PROXY_CONNECT_TIMEOUT_S, 12.0), cache_seconds=30,
headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8", headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8",
"Referer": "https://mdotjboss.state.mi.us/"}) "Referer": "https://mdotjboss.state.mi.us/"})
if host in {"publicstreamer1.cotrip.org", "publicstreamer2.cotrip.org", if host in {"publicstreamer1.cotrip.org", "publicstreamer2.cotrip.org",
"publicstreamer3.cotrip.org", "publicstreamer4.cotrip.org"}: "publicstreamer3.cotrip.org", "publicstreamer4.cotrip.org"}:
return _CCTVProxyProfile(name="cotrip-hls", timeout=(5.0, 20.0), cache_seconds=10, return _CCTVProxyProfile(name="cotrip-hls", timeout=(_CCTV_PROXY_CONNECT_TIMEOUT_S, 20.0), cache_seconds=10,
headers={"Accept": "application/vnd.apple.mpegurl,application/x-mpegURL,video/*,*/*;q=0.8", headers={"Accept": "application/vnd.apple.mpegurl,application/x-mpegURL,video/*,*/*;q=0.8",
"Referer": "https://www.cotrip.org/"}) "Referer": "https://www.cotrip.org/"})
if host == "cocam.carsprogram.org": if host == "cocam.carsprogram.org":
return _CCTVProxyProfile(name="cotrip-preview", timeout=(5.0, 12.0), cache_seconds=20, return _CCTVProxyProfile(name="cotrip-preview", timeout=(_CCTV_PROXY_CONNECT_TIMEOUT_S, 12.0), cache_seconds=20,
headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8", headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8",
"Referer": "https://www.cotrip.org/"}) "Referer": "https://www.cotrip.org/"})
if host in {"tripcheck.com", "www.tripcheck.com"}: if host in {"tripcheck.com", "www.tripcheck.com"}:
return _CCTVProxyProfile(name="odot-tripcheck", timeout=(5.0, 12.0), cache_seconds=30, return _CCTVProxyProfile(name="odot-tripcheck", timeout=(_CCTV_PROXY_CONNECT_TIMEOUT_S, 12.0), cache_seconds=30,
headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8"}) headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8"})
if host == "infocar.dgt.es": if host == "infocar.dgt.es":
return _CCTVProxyProfile(name="dgt-spain", timeout=(5.0, 8.0), cache_seconds=60, return _CCTVProxyProfile(name="dgt-spain", timeout=(_CCTV_PROXY_CONNECT_TIMEOUT_S, 8.0), cache_seconds=60,
headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8", headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8",
"Referer": "https://infocar.dgt.es/"}) "Referer": "https://infocar.dgt.es/"})
if host == "informo.madrid.es": if host == "informo.madrid.es":
return _CCTVProxyProfile(name="madrid-city", timeout=(5.0, 12.0), cache_seconds=30, return _CCTVProxyProfile(name="madrid-city", timeout=(_CCTV_PROXY_CONNECT_TIMEOUT_S, 12.0), cache_seconds=30,
headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8", headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8",
"Referer": "https://informo.madrid.es/"}) "Referer": "https://informo.madrid.es/"})
if host == "www.windy.com": if host in {"www.windy.com", "imgproxy.windy.com"}:
return _CCTVProxyProfile(name="windy-webcams", timeout=(5.0, 12.0), cache_seconds=60, return _CCTVProxyProfile(name="windy-webcams", timeout=(_CCTV_PROXY_CONNECT_TIMEOUT_S, 12.0), cache_seconds=60,
headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8"}) headers={"Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8",
return _CCTVProxyProfile(name="generic-cctv", timeout=(5.0, 10.0), cache_seconds=30, "Referer": "https://www.windy.com/"})
return _CCTVProxyProfile(name="generic-cctv", timeout=(_CCTV_PROXY_CONNECT_TIMEOUT_S, 8.0), cache_seconds=30,
headers={"Accept": "*/*"}) headers={"Accept": "*/*"})
@@ -221,13 +239,40 @@ def _rewrite_cctv_hls_playlist(base_url: str, body: str) -> str:
return "\n".join(rewritten_lines) + ("\n" if body.endswith("\n") else "") return "\n".join(rewritten_lines) + ("\n" if body.endswith("\n") else "")
def _infer_cctv_media_type_from_url(target_url: str, content_type: str) -> str:
from urllib.parse import urlparse
clean_type = str(content_type or "").split(";", 1)[0].strip().lower()
if clean_type and clean_type not in {"application/octet-stream", "binary/octet-stream"}:
return content_type
path = str(urlparse(target_url).path or "").lower()
if path.endswith((".jpg", ".jpeg")):
return "image/jpeg"
if path.endswith(".png"):
return "image/png"
if path.endswith(".webp"):
return "image/webp"
if path.endswith(".gif"):
return "image/gif"
if path.endswith(".mp4"):
return "video/mp4"
if path.endswith((".m3u8", ".m3u")):
return "application/vnd.apple.mpegurl"
if path.endswith((".mjpg", ".mjpeg")):
return "multipart/x-mixed-replace"
return content_type or "application/octet-stream"
def _proxy_cctv_media_response(request: Request, target_url: str): def _proxy_cctv_media_response(request: Request, target_url: str):
from urllib.parse import urlparse from urllib.parse import urlparse
from fastapi.responses import Response from fastapi.responses import Response
parsed = urlparse(target_url) parsed = urlparse(target_url)
profile = _cctv_proxy_profile_for_url(target_url) profile = _cctv_proxy_profile_for_url(target_url)
resp = _fetch_cctv_upstream_response(request, target_url, profile) resp = _fetch_cctv_upstream_response(request, target_url, profile)
content_type = resp.headers.get("Content-Type", "application/octet-stream") content_type = _infer_cctv_media_type_from_url(
target_url,
resp.headers.get("Content-Type", "application/octet-stream"),
)
is_hls_playlist = ( is_hls_playlist = (
".m3u8" in str(parsed.path or "").lower() ".m3u8" in str(parsed.path or "").lower()
or "mpegurl" in content_type.lower() or "mpegurl" in content_type.lower()
+153 -7
View File
@@ -185,11 +185,29 @@ def _bbox_spans(s, w, n, e) -> tuple:
return lat_span, max(0.0, lng_span) return lat_span, max(0.0, lng_span)
def _downsample_points(items: list, max_items: int) -> list: def _cap_startup_items(items: list | None, max_items: int) -> list:
if max_items <= 0 or len(items) <= max_items: if not items:
return []
if len(items) <= max_items:
return items return items
step = len(items) / float(max_items) return items[:max_items]
return [items[min(len(items) - 1, int(i * step))] for i in range(max_items)]
def _cap_fast_startup_payload(payload: dict) -> dict:
capped = dict(payload)
capped["commercial_flights"] = _cap_startup_items(capped.get("commercial_flights"), 800)
capped["private_flights"] = _cap_startup_items(capped.get("private_flights"), 300)
capped["private_jets"] = _cap_startup_items(capped.get("private_jets"), 150)
capped["ships"] = _cap_startup_items(capped.get("ships"), 1500)
capped["cctv"] = []
capped["sigint"] = _cap_startup_items(capped.get("sigint"), 500)
capped["trains"] = _cap_startup_items(capped.get("trains"), 100)
capped["startup_payload"] = True
return capped
def _cap_fast_dashboard_payload(payload: dict) -> dict:
return payload
def _world_and_continental_scale(has_bbox: bool, s, w, n, e) -> tuple: def _world_and_continental_scale(has_bbox: bool, s, w, n, e) -> tuple:
@@ -264,6 +282,20 @@ async def ais_feed(request: Request):
return {"status": "ok", "ingested": count} return {"status": "ok", "ingested": count}
@router.get("/api/trail/flight/{icao24}")
@limiter.limit("120/minute")
async def get_selected_flight_trail(icao24: str, request: Request): # noqa: ARG001
from services.fetchers.flights import get_flight_trail
return {"id": icao24, "trail": get_flight_trail(icao24)}
@router.get("/api/trail/ship/{mmsi}")
@limiter.limit("120/minute")
async def get_selected_ship_trail(mmsi: int, request: Request): # noqa: ARG001
from services.ais_stream import get_vessel_trail
return {"id": mmsi, "trail": get_vessel_trail(mmsi)}
@router.post("/api/viewport") @router.post("/api/viewport")
@limiter.limit("60/minute") @limiter.limit("60/minute")
async def update_viewport(vp: ViewportUpdate, request: Request): # noqa: ARG001 async def update_viewport(vp: ViewportUpdate, request: Request): # noqa: ARG001
@@ -306,8 +338,19 @@ async def update_layers(update: LayerUpdate, request: Request):
sigint_grid.mesh.stop() sigint_grid.mesh.stop()
logger.info("Meshtastic MQTT bridge stopped (layer disabled)") logger.info("Meshtastic MQTT bridge stopped (layer disabled)")
elif not old_mesh and new_mesh: elif not old_mesh and new_mesh:
sigint_grid.mesh.start() try:
logger.info("Meshtastic MQTT bridge started (layer enabled)") from services.meshtastic_mqtt_settings import mqtt_bridge_enabled
mqtt_enabled = mqtt_bridge_enabled()
except Exception:
mqtt_enabled = False
if mqtt_enabled:
sigint_grid.mesh.start()
logger.info("Meshtastic MQTT bridge started (layer enabled)")
else:
logger.info(
"Meshtastic layer enabled; MQTT bridge remains disabled "
"(set MESH_MQTT_ENABLED=true to participate in the public broker)"
)
if old_aprs and not new_aprs: if old_aprs and not new_aprs:
sigint_grid.aprs.stop() sigint_grid.aprs.stop()
logger.info("APRS bridge stopped (layer disabled)") logger.info("APRS bridge stopped (layer disabled)")
@@ -326,6 +369,104 @@ async def live_data(request: Request):
return get_latest_data() return get_latest_data()
@router.get("/api/bootstrap/critical")
@limiter.limit("180/minute")
async def bootstrap_critical(request: Request):
"""Cached first-paint payload for the dashboard.
This endpoint is intentionally memory-only: no upstream calls, no refresh,
and a bounded response. It exists so the map and threat feed can paint
before slower panels and background enrichers finish warming up.
"""
etag = _current_etag(prefix="bootstrap|critical|")
if request.headers.get("if-none-match") == etag:
return Response(status_code=304, headers={"ETag": etag, "Cache-Control": "no-cache"})
from services.fetchers._store import (
active_layers,
get_latest_data_subset_refs,
get_source_timestamps_snapshot,
)
d = get_latest_data_subset_refs(
"last_updated", "commercial_flights", "military_flights", "private_flights",
"private_jets", "tracked_flights", "ships", "uavs", "liveuamap", "gps_jamming",
"satellites", "satellite_source", "satellite_analysis", "sigint", "sigint_totals",
"trains", "news", "gdelt", "airports", "threat_level", "trending_markets",
"correlations", "fimi", "crowdthreat",
)
freshness = get_source_timestamps_snapshot()
ships_enabled = any(active_layers.get(key, True) for key in (
"ships_military", "ships_cargo", "ships_civilian", "ships_passenger", "ships_tracked_yachts"))
sigint_items = _filter_sigint_by_layers(d.get("sigint") or [], active_layers)
payload = {
"last_updated": d.get("last_updated"),
"commercial_flights": _cap_startup_items(
(d.get("commercial_flights") or []) if active_layers.get("flights", True) else [],
800,
),
"military_flights": _cap_startup_items(
(d.get("military_flights") or []) if active_layers.get("military", True) else [],
300,
),
"private_flights": _cap_startup_items(
(d.get("private_flights") or []) if active_layers.get("private", True) else [],
300,
),
"private_jets": _cap_startup_items(
(d.get("private_jets") or []) if active_layers.get("jets", True) else [],
150,
),
"tracked_flights": _cap_startup_items(
(d.get("tracked_flights") or []) if active_layers.get("tracked", True) else [],
250,
),
"ships": _cap_startup_items((d.get("ships") or []) if ships_enabled else [], 1500),
"uavs": _cap_startup_items((d.get("uavs") or []) if active_layers.get("military", True) else [], 100),
"liveuamap": _cap_startup_items(
(d.get("liveuamap") or []) if active_layers.get("global_incidents", True) else [],
300,
),
"gps_jamming": _cap_startup_items(
(d.get("gps_jamming") or []) if active_layers.get("gps_jamming", True) else [],
200,
),
"satellites": _cap_startup_items(
(d.get("satellites") or []) if active_layers.get("satellites", True) else [],
250,
),
"satellite_source": d.get("satellite_source", "none"),
"satellite_analysis": (d.get("satellite_analysis") or {}) if active_layers.get("satellites", True) else {},
"sigint": _cap_startup_items(
sigint_items if (active_layers.get("sigint_meshtastic", True) or active_layers.get("sigint_aprs", True)) else [],
500,
),
"sigint_totals": _sigint_totals_for_items(sigint_items),
"trains": _cap_startup_items((d.get("trains") or []) if active_layers.get("trains", True) else [], 100),
"news": _cap_startup_items(d.get("news") or [], 30),
"gdelt": _cap_startup_items((d.get("gdelt") or []) if active_layers.get("global_incidents", True) else [], 300),
"airports": _cap_startup_items(d.get("airports") or [], 500),
"threat_level": d.get("threat_level"),
"trending_markets": _cap_startup_items(d.get("trending_markets") or [], 10),
"correlations": _cap_startup_items(
(d.get("correlations") or []) if active_layers.get("correlations", True) else [],
50,
),
"fimi": d.get("fimi"),
"crowdthreat": _cap_startup_items(
(d.get("crowdthreat") or []) if active_layers.get("crowdthreat", True) else [],
150,
),
"freshness": freshness,
"bootstrap_ready": True,
"bootstrap_payload": True,
}
return Response(
content=orjson.dumps(_sanitize_payload(payload), default=str, option=orjson.OPT_NON_STR_KEYS),
media_type="application/json",
headers={"ETag": etag, "Cache-Control": "no-cache"},
)
@router.get("/api/live-data/fast") @router.get("/api/live-data/fast")
@limiter.limit("120/minute") @limiter.limit("120/minute")
async def live_data_fast( async def live_data_fast(
@@ -334,8 +475,9 @@ async def live_data_fast(
w: float = Query(None, description="West bound (ignored)", ge=-180, le=180), w: float = Query(None, description="West bound (ignored)", ge=-180, le=180),
n: float = Query(None, description="North bound (ignored)", ge=-90, le=90), n: float = Query(None, description="North bound (ignored)", ge=-90, le=90),
e: float = Query(None, description="East bound (ignored)", ge=-180, le=180), e: float = Query(None, description="East bound (ignored)", ge=-180, le=180),
initial: bool = Query(False, description="Return a capped startup payload for first paint"),
): ):
etag = _current_etag(prefix="fast|full|") etag = _current_etag(prefix="fast|initial|" if initial else "fast|full|")
if request.headers.get("if-none-match") == etag: if request.headers.get("if-none-match") == etag:
return Response(status_code=304, headers={"ETag": etag, "Cache-Control": "no-cache"}) return Response(status_code=304, headers={"ETag": etag, "Cache-Control": "no-cache"})
from services.fetchers._store import (active_layers, get_latest_data_subset_refs, get_source_timestamps_snapshot) from services.fetchers._store import (active_layers, get_latest_data_subset_refs, get_source_timestamps_snapshot)
@@ -371,6 +513,10 @@ async def live_data_fast(
"trains": (d.get("trains") or []) if active_layers.get("trains", True) else [], "trains": (d.get("trains") or []) if active_layers.get("trains", True) else [],
"freshness": freshness, "freshness": freshness,
} }
if initial:
payload = _cap_fast_startup_payload(payload)
else:
payload = _cap_fast_dashboard_payload(payload)
return Response(content=orjson.dumps(_sanitize_payload(payload)), media_type="application/json", return Response(content=orjson.dumps(_sanitize_payload(payload)), media_type="application/json",
headers={"ETag": etag, "Cache-Control": "no-cache"}) headers={"ETag": etag, "Cache-Control": "no-cache"})
+1 -1
View File
@@ -8,7 +8,7 @@ from services.data_fetcher import get_latest_data
from services.schemas import HealthResponse from services.schemas import HealthResponse
import os import os
APP_VERSION = os.environ.get("_HEALTH_APP_VERSION", "0.9.7") APP_VERSION = os.environ.get("_HEALTH_APP_VERSION", "0.9.75")
router = APIRouter() router = APIRouter()
+106
View File
@@ -754,6 +754,112 @@ async def mesh_send(request: Request):
} }
@router.post("/api/mesh/meshtastic/send", dependencies=[Depends(require_local_operator)])
@limiter.limit("10/minute")
@mesh_write_exempt(MeshWriteExemption.LOCAL_OPERATOR_ONLY)
async def meshtastic_public_send(request: Request):
"""Local public-MQTT send path for standalone Meshtastic-style identities."""
body = await request.json()
destination = str(body.get("destination", "") or "").strip() or "broadcast"
message = str(body.get("message", "") or "")
sender_id = str(body.get("sender_id", "") or "").strip().lower()
if not message:
return {"ok": False, "detail": "Missing required field: message"}
from services.mesh.mesh_router import (
MeshEnvelope,
MeshtasticTransport,
Priority,
TransportResult,
mesh_router,
)
from services.meshtastic_mqtt_settings import mqtt_bridge_enabled
if MeshtasticTransport._parse_node_id(sender_id) is None:
return {"ok": False, "detail": "Missing or invalid public Meshtastic address"}
if not mqtt_bridge_enabled():
return {"ok": False, "detail": "Meshtastic MQTT bridge is disabled"}
payload_bytes = len(message.encode("utf-8"))
payload_type = str(body.get("payload_type", "text") or "text")
max_bytes = _BYTE_LIMITS.get(payload_type, 200)
if payload_bytes > max_bytes:
return {
"ok": False,
"detail": f"Message too long ({payload_bytes} bytes). Maximum: {max_bytes} bytes for {payload_type} messages.",
}
priority_str = str(body.get("priority", "normal") or "normal").lower()
throttle_ok, throttle_reason = _check_throttle(sender_id, priority_str, "meshtastic")
if not throttle_ok:
return {"ok": False, "detail": throttle_reason}
priority_map = {
"emergency": Priority.EMERGENCY,
"high": Priority.HIGH,
"normal": Priority.NORMAL,
"low": Priority.LOW,
}
priority = priority_map.get(priority_str, Priority.NORMAL)
envelope = MeshEnvelope(
sender_id=sender_id,
destination=destination,
channel=str(body.get("channel", "LongFast") or "LongFast"),
priority=priority,
payload=message,
ephemeral=bool(body.get("ephemeral", False)),
trust_tier="public_degraded",
)
if not mesh_router.meshtastic.can_reach(envelope):
results = [TransportResult(False, "meshtastic", "Message exceeds Meshtastic payload limit")]
else:
cb_ok, cb_reason = mesh_router.breakers["meshtastic"].check_and_record(envelope.priority)
if not cb_ok:
results = [TransportResult(False, "meshtastic", cb_reason)]
else:
envelope.route_reason = (
"Local public Meshtastic MQTT path"
if MeshtasticTransport._parse_node_id(destination) is None
else "Local public Meshtastic direct node path"
)
credentials = {"mesh_region": str(body.get("mesh_region", "US") or "US")}
result = mesh_router.meshtastic.send(envelope, credentials)
if result.ok:
envelope.routed_via = mesh_router.meshtastic.NAME
results = [result]
any_ok = any(r.ok for r in results)
if any_ok and envelope.routed_via == "meshtastic":
try:
from datetime import datetime
from services.sigint_bridge import sigint_grid
bridge = sigint_grid.mesh
if bridge:
bridge.messages.appendleft(
{
"from": MeshtasticTransport.mesh_address_for_sender(sender_id),
"to": destination if MeshtasticTransport._parse_node_id(destination) is not None else "broadcast",
"text": message,
"region": str(body.get("mesh_region", "US") or "US"),
"channel": str(body.get("channel", "LongFast") or "LongFast"),
"timestamp": datetime.utcnow().isoformat() + "Z",
}
)
except Exception:
pass
return {
"ok": any_ok,
"message_id": envelope.message_id,
"event_id": "",
"routed_via": envelope.routed_via,
"route_reason": envelope.route_reason,
"results": [r.to_dict() for r in results],
}
@router.get("/api/mesh/log") @router.get("/api/mesh/log")
@limiter.limit("30/minute") @limiter.limit("30/minute")
async def mesh_log(request: Request): async def mesh_log(request: Request):
+24 -10
View File
@@ -869,21 +869,35 @@ async def api_get_wormhole_status(request: Request):
@router.post("/api/wormhole/join", dependencies=[Depends(require_local_operator)]) @router.post("/api/wormhole/join", dependencies=[Depends(require_local_operator)])
@limiter.limit("10/minute") @limiter.limit("10/minute")
async def api_wormhole_join(request: Request): async def api_wormhole_join(request: Request):
from services.config import get_settings
existing = read_wormhole_settings() existing = read_wormhole_settings()
updated = write_wormhole_settings( updated = write_wormhole_settings(
enabled=True, enabled=True,
transport="direct", transport="tor_arti",
socks_proxy="", socks_proxy=f"socks5h://127.0.0.1:{int(get_settings().MESH_ARTI_SOCKS_PORT or 9050)}",
socks_dns=True, socks_dns=True,
anonymous_mode=False, anonymous_mode=True,
) )
transport_changed = ( transport_changed = (
str(existing.get("transport", "direct")) != "direct" str(existing.get("transport", "direct")) != "tor_arti"
or str(existing.get("socks_proxy", "")) != "" or str(existing.get("socks_proxy", "")) != str(updated.get("socks_proxy", ""))
or bool(existing.get("socks_dns", True)) is not True or bool(existing.get("socks_dns", True)) is not True
or bool(existing.get("anonymous_mode", False)) is not False or bool(existing.get("anonymous_mode", False)) is not True
or bool(existing.get("enabled", False)) is not True or bool(existing.get("enabled", False)) is not True
) )
tor_result: dict[str, Any] = {"ok": False, "detail": "not started"}
try:
import asyncio
from routers.ai_intel import _write_env_value
from services.tor_hidden_service import tor_service
tor_result = await asyncio.to_thread(tor_service.start)
if tor_result.get("ok"):
_write_env_value("MESH_ARTI_ENABLED", "true")
get_settings.cache_clear()
except Exception as exc:
tor_result = {"ok": False, "detail": str(exc or type(exc).__name__)}
bootstrap_wormhole_identity() bootstrap_wormhole_identity()
bootstrap_wormhole_persona_state() bootstrap_wormhole_persona_state()
state = ( state = (
@@ -905,6 +919,7 @@ async def api_wormhole_join(request: Request):
"identity": get_transport_identity(), "identity": get_transport_identity(),
"runtime": state, "runtime": state,
"settings": updated, "settings": updated,
"tor": tor_result,
} }
@@ -914,10 +929,9 @@ async def api_wormhole_leave(request: Request):
updated = write_wormhole_settings(enabled=False) updated = write_wormhole_settings(enabled=False)
state = disconnect_wormhole(reason="leave_wormhole") state = disconnect_wormhole(reason="leave_wormhole")
# Disable node participation when the user leaves the Wormhole. # Leaving private DM mode must not disable Infonet participation. Infonet
from services.node_settings import write_node_settings # sync has its own private transport warmup and can remain connected to
# seed/peer nodes while MeshChat stays separately opt-in.
write_node_settings(enabled=False)
return { return {
"ok": True, "ok": True,
+85 -4
View File
@@ -17,6 +17,18 @@ AIS_WS_URL = "wss://stream.aisstream.io/v0/stream"
API_KEY = os.environ.get("AIS_API_KEY", "") API_KEY = os.environ.get("AIS_API_KEY", "")
def _env_truthy(name: str) -> bool:
return str(os.getenv(name, "")).strip().lower() in {"1", "true", "yes", "on"}
def ais_stream_proxy_enabled() -> bool:
"""Return whether the external Node AIS proxy may be started."""
setting = str(os.getenv("SHADOWBROKER_ENABLE_AIS_STREAM_PROXY", "")).strip().lower()
if setting:
return _env_truthy("SHADOWBROKER_ENABLE_AIS_STREAM_PROXY")
return True
# AIS vessel type code classification # AIS vessel type code classification
# See: https://coast.noaa.gov/data/marinecadastre/ais/VesselTypeCodes2018.pdf # See: https://coast.noaa.gov/data/marinecadastre/ais/VesselTypeCodes2018.pdf
def classify_vessel(ais_type: int, mmsi: int) -> str: def classify_vessel(ais_type: int, mmsi: int) -> str:
@@ -327,16 +339,61 @@ def get_country_from_mmsi(mmsi: int) -> str:
# Global vessel store: MMSI → vessel dict # Global vessel store: MMSI → vessel dict
_vessels: dict[int, dict] = {} _vessels: dict[int, dict] = {}
_vessel_trails: dict[int, dict] = {}
_vessels_lock = threading.Lock() _vessels_lock = threading.Lock()
_ws_thread: threading.Thread | None = None _ws_thread: threading.Thread | None = None
_ws_running = False _ws_running = False
_proxy_process = None _proxy_process = None
_VESSEL_TRAIL_INTERVAL_S = 120
_VESSEL_TRAIL_MAX_POINTS = 240
import os import os
CACHE_FILE = os.path.join(os.path.dirname(__file__), "ais_cache.json") CACHE_FILE = os.path.join(os.path.dirname(__file__), "ais_cache.json")
def _record_vessel_trail_locked(mmsi: int, lat, lng, sog=0, now_ts: float | None = None) -> None:
"""Append a sampled AIS trail point. Caller must hold _vessels_lock."""
if lat is None or lng is None:
return
try:
lat_f = float(lat)
lng_f = float(lng)
except (TypeError, ValueError):
return
if abs(lat_f) > 90 or abs(lng_f) > 180 or (lat_f == 0 and lng_f == 0):
return
now = now_ts or time.time()
trail_data = _vessel_trails.setdefault(int(mmsi), {"points": [], "last_seen": now})
point = [round(lat_f, 5), round(lng_f, 5), round(float(sog or 0), 1), round(now)]
last_point_ts = trail_data["points"][-1][3] if trail_data["points"] else 0
if now - last_point_ts < _VESSEL_TRAIL_INTERVAL_S:
trail_data["last_seen"] = now
return
if (
trail_data["points"]
and trail_data["points"][-1][0] == point[0]
and trail_data["points"][-1][1] == point[1]
):
trail_data["last_seen"] = now
return
trail_data["points"].append(point)
trail_data["last_seen"] = now
if len(trail_data["points"]) > _VESSEL_TRAIL_MAX_POINTS:
trail_data["points"] = trail_data["points"][-_VESSEL_TRAIL_MAX_POINTS:]
def get_vessel_trail(mmsi: int) -> list:
"""Return the accumulated trail for a single vessel without expanding live payloads."""
try:
key = int(mmsi)
except (TypeError, ValueError):
return []
with _vessels_lock:
points = _vessel_trails.get(key, {}).get("points", [])
return [list(point) for point in points]
def _save_cache(): def _save_cache():
"""Save vessel data to disk for persistence across restarts.""" """Save vessel data to disk for persistence across restarts."""
try: try:
@@ -379,6 +436,7 @@ def prune_stale_vessels():
stale_keys = [k for k, v in _vessels.items() if v.get("_updated", 0) < stale_cutoff] stale_keys = [k for k, v in _vessels.items() if v.get("_updated", 0) < stale_cutoff]
for k in stale_keys: for k in stale_keys:
del _vessels[k] del _vessels[k]
_vessel_trails.pop(k, None)
if stale_keys: if stale_keys:
logger.info(f"AIS pruned {len(stale_keys)} stale vessels") logger.info(f"AIS pruned {len(stale_keys)} stale vessels")
@@ -447,6 +505,7 @@ def ingest_ais_catcher(msgs: list[dict]) -> int:
heading = msg.get("heading", 511) heading = msg.get("heading", 511)
vessel["heading"] = heading if heading != 511 else vessel.get("cog", 0) vessel["heading"] = heading if heading != 511 else vessel.get("cog", 0)
vessel["_updated"] = now vessel["_updated"] = now
_record_vessel_trail_locked(mmsi, lat, lon, vessel["sog"], now)
if msg.get("shipname"): if msg.get("shipname"):
vessel["name"] = msg["shipname"].strip() vessel["name"] = msg["shipname"].strip()
count += 1 count += 1
@@ -496,6 +555,12 @@ def _ais_stream_loop():
logger.info("Starting Node.js AIS Stream Proxy...") logger.info("Starting Node.js AIS Stream Proxy...")
proxy_env = os.environ.copy() proxy_env = os.environ.copy()
proxy_env["AIS_API_KEY"] = API_KEY proxy_env["AIS_API_KEY"] = API_KEY
popen_kwargs = {}
if os.name == "nt":
popen_kwargs["creationflags"] = (
getattr(subprocess, "CREATE_NO_WINDOW", 0)
| getattr(subprocess, "CREATE_NEW_PROCESS_GROUP", 0)
)
process = subprocess.Popen( process = subprocess.Popen(
["node", proxy_script], ["node", proxy_script],
stdin=subprocess.PIPE, stdin=subprocess.PIPE,
@@ -504,6 +569,7 @@ def _ais_stream_loop():
text=True, text=True,
bufsize=1, bufsize=1,
env=proxy_env, env=proxy_env,
**popen_kwargs,
) )
with _vessels_lock: with _vessels_lock:
_proxy_process = process _proxy_process = process
@@ -576,7 +642,9 @@ def _ais_stream_loop():
vessel["cog"] = report.get("Cog", 0) vessel["cog"] = report.get("Cog", 0)
heading = report.get("TrueHeading", 511) heading = report.get("TrueHeading", 511)
vessel["heading"] = heading if heading != 511 else report.get("Cog", 0) vessel["heading"] = heading if heading != 511 else report.get("Cog", 0)
vessel["_updated"] = time.time() now_ts = time.time()
vessel["_updated"] = now_ts
_record_vessel_trail_locked(mmsi, lat, lng, vessel["sog"], now_ts)
# Use metadata name if we don't have one yet # Use metadata name if we don't have one yet
if not vessel.get("name") or vessel["name"] == "UNKNOWN": if not vessel.get("name") or vessel["name"] == "UNKNOWN":
vessel["name"] = ( vessel["name"] = (
@@ -646,6 +714,22 @@ def _run_ais_loop():
def start_ais_stream(): def start_ais_stream():
"""Start the AIS WebSocket stream in a background thread.""" """Start the AIS WebSocket stream in a background thread."""
global _ws_thread, _ws_running global _ws_thread, _ws_running
# Always load cached vessel data first so the ships layer can paint even
# when live streaming is disabled or the upstream is unavailable.
_load_cache()
if not API_KEY:
logger.info("AIS_API_KEY not set — ship tracking disabled. Set AIS_API_KEY to enable.")
return
if not ais_stream_proxy_enabled():
logger.info(
"AIS live stream proxy disabled for this runtime; using cached AIS vessels. "
"Set SHADOWBROKER_ENABLE_AIS_STREAM_PROXY=1 to opt in."
)
return
with _vessels_lock: with _vessels_lock:
if _ws_running: if _ws_running:
logger.info("AIS Stream already running") logger.info("AIS Stream already running")
@@ -656,9 +740,6 @@ def start_ais_stream():
logger.info("AIS Stream already running") logger.info("AIS Stream already running")
return return
# Load cached vessel data from disk
_load_cache()
_ws_thread = threading.Thread(target=_run_ais_loop, daemon=True, name="ais-stream") _ws_thread = threading.Thread(target=_run_ais_loop, daemon=True, name="ais-stream")
_ws_thread.start() _ws_thread.start()
logger.info("AIS Stream background thread started") logger.info("AIS Stream background thread started")
+147
View File
@@ -4,12 +4,21 @@ Keys are stored in the backend .env file and loaded via python-dotenv.
""" """
import os import os
import re
import tempfile
from pathlib import Path from pathlib import Path
# Path to the backend .env file # Path to the backend .env file
ENV_PATH = Path(__file__).parent.parent / ".env" ENV_PATH = Path(__file__).parent.parent / ".env"
# Path to the example template that ships with the repo # Path to the example template that ships with the repo
ENV_EXAMPLE_PATH = Path(__file__).parent.parent.parent / ".env.example" ENV_EXAMPLE_PATH = Path(__file__).parent.parent.parent / ".env.example"
DATA_DIR = Path(os.environ.get("SB_DATA_DIR", str(Path(__file__).parent.parent / "data")))
if not DATA_DIR.is_absolute():
DATA_DIR = Path(__file__).parent.parent / DATA_DIR
OPERATOR_KEYS_ENV_PATH = Path(
os.environ.get("SHADOWBROKER_OPERATOR_KEYS_ENV", str(DATA_DIR / "operator_api_keys.env"))
)
_ENV_KEY_RE = re.compile(r"^[A-Z][A-Z0-9_]*$")
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# API Registry — every external service the dashboard depends on # API Registry — every external service the dashboard depends on
@@ -143,6 +152,85 @@ API_REGISTRY = [
}, },
] ]
ALLOWED_ENV_KEYS = {
str(api["env_key"])
for api in API_REGISTRY
if api.get("env_key")
}
def _parse_env_file(path: Path) -> dict[str, str]:
values: dict[str, str] = {}
if not path.exists():
return values
try:
text = path.read_text(encoding="utf-8")
except OSError:
return values
for raw_line in text.splitlines():
line = raw_line.strip()
if not line or line.startswith("#") or "=" not in line:
continue
key, value = line.split("=", 1)
key = key.strip()
if not _ENV_KEY_RE.match(key):
continue
value = value.strip()
if len(value) >= 2 and value[0] == value[-1] and value[0] in {"'", '"'}:
value = value[1:-1]
values[key] = value
return values
def _quote_env_value(value: str) -> str:
escaped = value.replace("\\", "\\\\").replace('"', '\\"')
return f'"{escaped}"'
def _write_env_values(path: Path, updates: dict[str, str]) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
lines = path.read_text(encoding="utf-8").splitlines() if path.exists() else []
seen: set[str] = set()
next_lines: list[str] = []
for raw_line in lines:
stripped = raw_line.strip()
if "=" not in stripped or stripped.startswith("#"):
next_lines.append(raw_line)
continue
key = stripped.split("=", 1)[0].strip()
if key in updates:
next_lines.append(f"{key}={_quote_env_value(updates[key])}")
seen.add(key)
else:
next_lines.append(raw_line)
for key, value in updates.items():
if key not in seen:
next_lines.append(f"{key}={_quote_env_value(value)}")
fd, tmp_name = tempfile.mkstemp(dir=str(path.parent), prefix=f"{path.name}.tmp.", text=True)
tmp_path = Path(tmp_name)
try:
with os.fdopen(fd, "w", encoding="utf-8", newline="\n") as handle:
handle.write("\n".join(next_lines).rstrip() + "\n")
if os.name != "nt":
os.chmod(tmp_path, 0o600)
os.replace(tmp_path, path)
if os.name != "nt":
os.chmod(path, 0o600)
finally:
try:
if tmp_path.exists():
tmp_path.unlink()
except OSError:
pass
def load_persisted_api_keys_into_environ() -> None:
"""Load persisted operator API keys if no process env value exists."""
for key, value in _parse_env_file(OPERATOR_KEYS_ENV_PATH).items():
if key in ALLOWED_ENV_KEYS and value and not os.environ.get(key):
os.environ[key] = value
def get_env_path_info() -> dict: def get_env_path_info() -> dict:
"""Return absolute paths for the backend .env and .env.example template. """Return absolute paths for the backend .env and .env.example template.
@@ -160,6 +248,10 @@ def get_env_path_info() -> dict:
and (not env_path.exists() or os.access(env_path, os.W_OK)), and (not env_path.exists() or os.access(env_path, os.W_OK)),
"env_example_path": str(example_path), "env_example_path": str(example_path),
"env_example_path_exists": example_path.exists(), "env_example_path_exists": example_path.exists(),
"operator_keys_env_path": str(OPERATOR_KEYS_ENV_PATH.resolve()),
"operator_keys_env_path_exists": OPERATOR_KEYS_ENV_PATH.exists(),
"operator_keys_env_path_writable": os.access(OPERATOR_KEYS_ENV_PATH.parent, os.W_OK)
and (not OPERATOR_KEYS_ENV_PATH.exists() or os.access(OPERATOR_KEYS_ENV_PATH, os.W_OK)),
} }
@@ -171,6 +263,7 @@ def get_api_keys():
`is_set` to render a CONFIGURED / NOT CONFIGURED badge and the path `is_set` to render a CONFIGURED / NOT CONFIGURED badge and the path
info from `get_env_path_info()` to tell them where to put each key. info from `get_env_path_info()` to tell them where to put each key.
""" """
load_persisted_api_keys_into_environ()
result = [] result = []
for api in API_REGISTRY: for api in API_REGISTRY:
entry = { entry = {
@@ -189,3 +282,57 @@ def get_api_keys():
entry["is_set"] = bool(raw) entry["is_set"] = bool(raw)
result.append(entry) result.append(entry)
return result return result
def save_api_keys(updates: dict[str, str]) -> dict:
"""Persist allowed API keys from a local operator request.
Values are accepted write-only: the response includes only configured flags.
"""
clean: dict[str, str] = {}
for key, value in updates.items():
env_key = str(key or "").strip().upper()
if env_key not in ALLOWED_ENV_KEYS:
continue
clean_value = str(value or "").strip()
if clean_value:
clean[env_key] = clean_value
if not clean:
return {"ok": False, "detail": "No supported API keys were provided."}
_write_env_values(OPERATOR_KEYS_ENV_PATH, clean)
try:
_write_env_values(ENV_PATH, clean)
except OSError:
# The persistent operator key file is the source of truth for Docker.
pass
for key, value in clean.items():
os.environ[key] = value
if "AIS_API_KEY" in clean:
try:
from services import ais_stream
ais_stream.API_KEY = clean["AIS_API_KEY"]
except Exception:
pass
if "OPENSKY_CLIENT_ID" in clean or "OPENSKY_CLIENT_SECRET" in clean:
try:
from services.fetchers import flights
flights.opensky_client.client_id = os.environ.get("OPENSKY_CLIENT_ID", "")
flights.opensky_client.client_secret = os.environ.get("OPENSKY_CLIENT_SECRET", "")
flights.opensky_client.token = None
flights.opensky_client.expires_at = 0
except Exception:
pass
try:
from services.config import get_settings
get_settings.cache_clear()
except Exception:
pass
return {
"ok": True,
"updated": sorted(clean.keys()),
"keys": get_api_keys(),
"env": get_env_path_info(),
}
+14 -1
View File
@@ -32,7 +32,14 @@ class Settings(BaseSettings):
MESH_ARTI_ENABLED: bool = False MESH_ARTI_ENABLED: bool = False
MESH_ARTI_SOCKS_PORT: int = 9050 MESH_ARTI_SOCKS_PORT: int = 9050
MESH_RELAY_PEERS: str = "" MESH_RELAY_PEERS: str = ""
MESH_DEFAULT_SYNC_PEERS: str = "https://node.shadowbroker.info" # Bootstrap seeds are discovery hints, not authoritative network roots.
# Nodes promote healthy discovered peers from the store/manifest over time.
MESH_BOOTSTRAP_SEED_PEERS: str = "http://gqpbunqbgtkcqilvclm3xrkt3zowjyl3s62kkktvojgvxzizamvbrqid.onion:8000"
# Legacy name kept for older compose/.env files.
MESH_DEFAULT_SYNC_PEERS: str = ""
# Infonet/Wormhole must fail closed to private transports by default.
# Set true only for local relay development or explicitly public testnets.
MESH_INFONET_ALLOW_CLEARNET_SYNC: bool = False
MESH_BOOTSTRAP_DISABLED: bool = False MESH_BOOTSTRAP_DISABLED: bool = False
MESH_BOOTSTRAP_MANIFEST_PATH: str = "data/bootstrap_peers.json" MESH_BOOTSTRAP_MANIFEST_PATH: str = "data/bootstrap_peers.json"
MESH_BOOTSTRAP_SIGNER_PUBLIC_KEY: str = "" MESH_BOOTSTRAP_SIGNER_PUBLIC_KEY: str = ""
@@ -210,6 +217,7 @@ class Settings(BaseSettings):
MESH_ALLOW_RAW_SECURE_STORAGE_FALLBACK: bool = False MESH_ALLOW_RAW_SECURE_STORAGE_FALLBACK: bool = False
MESH_ACK_RAW_FALLBACK_AT_OWN_RISK: bool = False MESH_ACK_RAW_FALLBACK_AT_OWN_RISK: bool = False
MESH_SECURE_STORAGE_SECRET: str = "" MESH_SECURE_STORAGE_SECRET: str = ""
MESH_SECURE_STORAGE_SECRET_FILE: str = ""
MESH_PRIVATE_LOG_TTL_S: int = 900 MESH_PRIVATE_LOG_TTL_S: int = 900
# Sprint 1 rollout: restored DM boot probes stay disabled by default until # Sprint 1 rollout: restored DM boot probes stay disabled by default until
# the architect reviews false positives from the observe-only path. # the architect reviews false positives from the observe-only path.
@@ -302,6 +310,11 @@ class Settings(BaseSettings):
@lru_cache @lru_cache
def get_settings() -> Settings: def get_settings() -> Settings:
try:
from services.api_settings import load_persisted_api_keys_into_environ
load_persisted_api_keys_into_environ()
except Exception:
pass
return Settings() return Settings()
+268 -44
View File
@@ -19,6 +19,7 @@ import concurrent.futures
import json import json
import math import math
import os import os
import threading
import time import time
from datetime import datetime, timedelta from datetime import datetime, timedelta
from pathlib import Path from pathlib import Path
@@ -105,7 +106,7 @@ _SLOW_FETCH_S = float(os.environ.get("FETCH_SLOW_THRESHOLD_S", "5"))
# Hard wall-clock limit per individual fetch task. A task that exceeds this # Hard wall-clock limit per individual fetch task. A task that exceeds this
# is treated as a failure so it cannot block an entire fetch tier indefinitely. # is treated as a failure so it cannot block an entire fetch tier indefinitely.
_TASK_HARD_TIMEOUT_S = float(os.environ.get("FETCH_TASK_TIMEOUT_S", "120")) _TASK_HARD_TIMEOUT_S = float(os.environ.get("FETCH_TASK_TIMEOUT_S", "120"))
_FAST_STARTUP_CACHE_MAX_AGE_S = float(os.environ.get("FAST_STARTUP_CACHE_MAX_AGE_S", "300")) _FAST_STARTUP_CACHE_MAX_AGE_S = float(os.environ.get("FAST_STARTUP_CACHE_MAX_AGE_S", "21600"))
_FAST_STARTUP_CACHE_PATH = Path(__file__).resolve().parents[1] / "data" / "fast_startup_cache.json" _FAST_STARTUP_CACHE_PATH = Path(__file__).resolve().parents[1] / "data" / "fast_startup_cache.json"
_FAST_STARTUP_CACHE_KEYS = ( _FAST_STARTUP_CACHE_KEYS = (
"commercial_flights", "commercial_flights",
@@ -123,10 +124,32 @@ _FAST_STARTUP_CACHE_KEYS = (
"sigint_totals", "sigint_totals",
"trains", "trains",
) )
_INTEL_STARTUP_CACHE_MAX_AGE_S = float(os.environ.get("INTEL_STARTUP_CACHE_MAX_AGE_S", "21600"))
_INTEL_STARTUP_CACHE_PATH = Path(__file__).resolve().parents[1] / "data" / "intel_startup_cache.json"
_INTEL_STARTUP_CACHE_KEYS = (
"news",
"gdelt",
"liveuamap",
"threat_level",
"trending_markets",
"correlations",
"fimi",
"crowdthreat",
"uap_sightings",
"military_bases",
"wastewater",
)
_STARTUP_PRIORITY_TIMEOUT_S = float(os.environ.get("SHADOWBROKER_STARTUP_PRIORITY_TIMEOUT_S", "18"))
_STARTUP_HEAVY_REFRESH_DELAY_S = float(os.environ.get("SHADOWBROKER_STARTUP_HEAVY_REFRESH_DELAY_S", "90"))
_STARTUP_HEAVY_REFRESH_STARTED = False
_STARTUP_HEAVY_REFRESH_LOCK = threading.Lock()
_FETCH_WORKERS = int(os.environ.get("SHADOWBROKER_FETCH_WORKERS", "8"))
_SLOW_FETCH_CONCURRENCY = int(os.environ.get("SHADOWBROKER_SLOW_FETCH_CONCURRENCY", "4"))
_STARTUP_HEAVY_CONCURRENCY = int(os.environ.get("SHADOWBROKER_STARTUP_HEAVY_CONCURRENCY", "2"))
# Shared thread pool — reused across all fetch cycles instead of creating/destroying per tick # Shared thread pool — reused across all fetch cycles instead of creating/destroying per tick
_SHARED_EXECUTOR = concurrent.futures.ThreadPoolExecutor( _SHARED_EXECUTOR = concurrent.futures.ThreadPoolExecutor(
max_workers=20, thread_name_prefix="fetch" max_workers=max(2, _FETCH_WORKERS), thread_name_prefix="fetch"
) )
@@ -140,6 +163,14 @@ def _cache_json_safe(value):
return value return value
def _has_cache_value(value) -> bool:
if value is None:
return False
if isinstance(value, (list, tuple, dict, set)):
return bool(value)
return True
def _load_fast_startup_cache_if_available() -> bool: def _load_fast_startup_cache_if_available() -> bool:
"""Seed moving layers from a recent disk cache while live fetches warm up.""" """Seed moving layers from a recent disk cache while live fetches warm up."""
if _FAST_STARTUP_CACHE_MAX_AGE_S <= 0 or not _FAST_STARTUP_CACHE_PATH.exists(): if _FAST_STARTUP_CACHE_MAX_AGE_S <= 0 or not _FAST_STARTUP_CACHE_PATH.exists():
@@ -184,10 +215,15 @@ def _save_fast_startup_cache() -> None:
"""Persist recent moving layers for the next cold start.""" """Persist recent moving layers for the next cold start."""
try: try:
with _data_lock: with _data_lock:
layers = {
key: latest_data.get(key)
for key in _FAST_STARTUP_CACHE_KEYS
if _has_cache_value(latest_data.get(key))
}
payload = { payload = {
"cached_at": time.time(), "cached_at": time.time(),
"last_updated": latest_data.get("last_updated"), "last_updated": latest_data.get("last_updated"),
"layers": {key: latest_data.get(key) for key in _FAST_STARTUP_CACHE_KEYS}, "layers": layers,
"freshness": { "freshness": {
key: source_timestamps.get(key) key: source_timestamps.get(key)
for key in _FAST_STARTUP_CACHE_KEYS for key in _FAST_STARTUP_CACHE_KEYS
@@ -204,14 +240,106 @@ def _save_fast_startup_cache() -> None:
logger.debug("Fast startup cache save skipped: %s", e) logger.debug("Fast startup cache save skipped: %s", e)
def _load_intel_startup_cache_if_available() -> bool:
"""Seed the right-side intelligence panel from disk while live feeds warm up."""
if _INTEL_STARTUP_CACHE_MAX_AGE_S <= 0 or not _INTEL_STARTUP_CACHE_PATH.exists():
return False
try:
with _INTEL_STARTUP_CACHE_PATH.open("r", encoding="utf-8") as fh:
payload = json.load(fh)
cached_at = float(payload.get("cached_at") or 0)
age_s = time.time() - cached_at
if cached_at <= 0 or age_s > _INTEL_STARTUP_CACHE_MAX_AGE_S:
logger.info("Skipping stale intel startup cache (age %.1fs)", age_s)
return False
layers = payload.get("layers") or {}
freshness = payload.get("freshness") or {}
loaded: list[str] = []
with _data_lock:
for key in _INTEL_STARTUP_CACHE_KEYS:
if key in layers:
latest_data[key] = layers[key]
loaded.append(key)
for key, ts in freshness.items():
source_timestamps[str(key)] = ts
if payload.get("last_updated"):
latest_data["last_updated"] = payload.get("last_updated")
if not loaded:
return False
from services.fetchers._store import bump_data_version
bump_data_version()
logger.info(
"Loaded intel startup cache for %d layers (age %.1fs) so Global Threat Intercept can paint early",
len(loaded),
age_s,
)
return True
except Exception as e:
logger.warning("Intel startup cache load failed (non-fatal): %s", e)
return False
def _save_intel_startup_cache() -> None:
"""Persist compact right-side intelligence data for the next cold start."""
try:
with _data_lock:
layers = {
key: latest_data.get(key)
for key in _INTEL_STARTUP_CACHE_KEYS
if _has_cache_value(latest_data.get(key))
}
payload = {
"cached_at": time.time(),
"last_updated": latest_data.get("last_updated"),
"layers": layers,
"freshness": {
key: source_timestamps.get(key)
for key in _INTEL_STARTUP_CACHE_KEYS
if source_timestamps.get(key)
},
}
safe_payload = _cache_json_safe(payload)
_INTEL_STARTUP_CACHE_PATH.parent.mkdir(parents=True, exist_ok=True)
tmp_path = _INTEL_STARTUP_CACHE_PATH.with_suffix(".tmp")
with tmp_path.open("w", encoding="utf-8") as fh:
json.dump(safe_payload, fh, separators=(",", ":"))
tmp_path.replace(_INTEL_STARTUP_CACHE_PATH)
except Exception as e:
logger.debug("Intel startup cache save skipped: %s", e)
def seed_startup_caches() -> None:
"""Load disk-backed first-paint caches without touching remote services."""
load_meshtastic_cache_if_available()
_load_fast_startup_cache_if_available()
_load_intel_startup_cache_if_available()
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Scheduler & Orchestration # Scheduler & Orchestration
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
def _run_tasks(label: str, funcs: list): def _run_tasks(label: str, funcs: list, *, max_concurrency: int | None = None):
"""Run tasks concurrently and log any exceptions (do not fail silently).""" """Run tasks concurrently and log any exceptions (do not fail silently)."""
if not funcs: if not funcs:
return return
futures = {_SHARED_EXECUTOR.submit(func): (func.__name__, time.perf_counter()) for func in funcs} if max_concurrency is None:
if label.startswith("slow-tier"):
max_concurrency = _SLOW_FETCH_CONCURRENCY
elif label.startswith("startup-heavy"):
max_concurrency = _STARTUP_HEAVY_CONCURRENCY
else:
max_concurrency = len(funcs)
max_concurrency = max(1, min(max_concurrency, len(funcs)))
remaining_funcs = list(funcs)
while remaining_funcs:
batch, remaining_funcs = remaining_funcs[:max_concurrency], remaining_funcs[max_concurrency:]
futures = {_SHARED_EXECUTOR.submit(func): (func.__name__, time.perf_counter()) for func in batch}
_drain_task_futures(label, futures)
def _drain_task_futures(label: str, futures: dict):
# Iterate directly so future.result(timeout=...) is the blocking call. # Iterate directly so future.result(timeout=...) is the blocking call.
# as_completed() blocks inside __next__() waiting for completion — the timeout # as_completed() blocks inside __next__() waiting for completion — the timeout
# on result() would never be reached for a hanging task under that pattern. # on result() would never be reached for a hanging task under that pattern.
@@ -262,7 +390,6 @@ def update_fast_data():
fetch_satellites, fetch_satellites,
fetch_sigint, fetch_sigint,
fetch_trains, fetch_trains,
fetch_tinygs,
] ]
_run_tasks("fast-tier", fast_funcs) _run_tasks("fast-tier", fast_funcs)
with _data_lock: with _data_lock:
@@ -289,6 +416,7 @@ def update_slow_data():
fetch_cctv, fetch_cctv,
fetch_kiwisdr, fetch_kiwisdr,
fetch_satnogs, fetch_satnogs,
fetch_tinygs,
fetch_frontlines, fetch_frontlines,
fetch_datacenters, fetch_datacenters,
fetch_military_bases, fetch_military_bases,
@@ -313,9 +441,76 @@ def update_slow_data():
logger.error("Correlation engine failed: %s", e) logger.error("Correlation engine failed: %s", e)
from services.fetchers._store import bump_data_version from services.fetchers._store import bump_data_version
bump_data_version() bump_data_version()
_save_intel_startup_cache()
logger.info("Slow-tier update complete.") logger.info("Slow-tier update complete.")
def _record_fetch_success(label: str, name: str, start: float) -> None:
duration = time.perf_counter() - start
from services.fetch_health import record_success
record_success(name, duration_s=duration)
if duration > _SLOW_FETCH_S:
logger.warning(f"{label} task slow: {name} took {duration:.2f}s")
def _record_fetch_failure(label: str, name: str, start: float, error: Exception) -> None:
duration = time.perf_counter() - start
from services.fetch_health import record_failure
record_failure(name, error=error, duration_s=duration)
logger.exception(f"{label} task failed: {name}")
def _load_cctv_cache_for_startup() -> None:
"""Load cached CCTV rows without running remote ingestors during first paint."""
try:
fetch_cctv()
except Exception as e:
logger.warning("Startup CCTV cache load failed (non-fatal): %s", e)
def _run_delayed_startup_heavy_refresh() -> None:
if _STARTUP_HEAVY_REFRESH_DELAY_S > 0:
logger.info(
"Startup heavy synthesis delayed %.0fs so the dashboard can finish first paint",
_STARTUP_HEAVY_REFRESH_DELAY_S,
)
time.sleep(_STARTUP_HEAVY_REFRESH_DELAY_S)
logger.info("Startup heavy synthesis beginning (slow feeds, enrichment, daily products)...")
_run_tasks(
"startup-heavy",
[
update_slow_data,
fetch_volcanoes,
fetch_viirs_change_nodes,
fetch_unusual_whales,
fetch_fimi,
fetch_uap_sightings,
fetch_wastewater,
fetch_sar_catalog,
fetch_sar_products,
],
)
logger.info("Startup heavy synthesis complete.")
def _schedule_delayed_startup_heavy_refresh() -> None:
global _STARTUP_HEAVY_REFRESH_STARTED
if _STARTUP_HEAVY_REFRESH_DELAY_S < 0:
logger.info("Startup heavy synthesis disabled by SHADOWBROKER_STARTUP_HEAVY_REFRESH_DELAY_S")
return
with _STARTUP_HEAVY_REFRESH_LOCK:
if _STARTUP_HEAVY_REFRESH_STARTED:
return
_STARTUP_HEAVY_REFRESH_STARTED = True
threading.Thread(
target=_run_delayed_startup_heavy_refresh,
name="startup-heavy-refresh",
daemon=True,
).start()
def update_all_data(*, startup_mode: bool = False): def update_all_data(*, startup_mode: bool = False):
"""Full refresh. """Full refresh.
@@ -324,50 +519,79 @@ def update_all_data(*, startup_mode: bool = False):
""" """
logger.info("Full data update starting (parallel)...") logger.info("Full data update starting (parallel)...")
# Preload Meshtastic map cache immediately (instant, from disk) # Preload Meshtastic map cache immediately (instant, from disk)
load_meshtastic_cache_if_available() seed_startup_caches()
_load_fast_startup_cache_if_available()
with _data_lock: with _data_lock:
meshtastic_seeded = bool(latest_data.get("meshtastic_map_nodes")) meshtastic_seeded = bool(latest_data.get("meshtastic_map_nodes"))
futures = { if startup_mode:
_SHARED_EXECUTOR.submit(fetch_airports): ("fetch_airports", time.perf_counter()), _load_cctv_cache_for_startup()
_SHARED_EXECUTOR.submit(update_fast_data): ("update_fast_data", time.perf_counter()), priority_funcs = [
_SHARED_EXECUTOR.submit(update_slow_data): ("update_slow_data", time.perf_counter()), fetch_airports,
_SHARED_EXECUTOR.submit(fetch_volcanoes): ("fetch_volcanoes", time.perf_counter()), update_fast_data,
_SHARED_EXECUTOR.submit(fetch_viirs_change_nodes): ("fetch_viirs_change_nodes", time.perf_counter()), fetch_news,
_SHARED_EXECUTOR.submit(fetch_unusual_whales): ("fetch_unusual_whales", time.perf_counter()), fetch_gdelt,
_SHARED_EXECUTOR.submit(fetch_fimi): ("fetch_fimi", time.perf_counter()), fetch_crowdthreat,
_SHARED_EXECUTOR.submit(fetch_gdelt): ("fetch_gdelt", time.perf_counter()), fetch_firms_fires,
_SHARED_EXECUTOR.submit(update_liveuamap): ("update_liveuamap", time.perf_counter()), fetch_weather_alerts,
_SHARED_EXECUTOR.submit(fetch_uap_sightings): ("fetch_uap_sightings", time.perf_counter()), ]
_SHARED_EXECUTOR.submit(fetch_wastewater): ("fetch_wastewater", time.perf_counter()), if not meshtastic_seeded:
_SHARED_EXECUTOR.submit(fetch_crowdthreat): ("fetch_crowdthreat", time.perf_counter()), priority_funcs.append(fetch_meshtastic_nodes)
_SHARED_EXECUTOR.submit(fetch_sar_catalog): ("fetch_sar_catalog", time.perf_counter()), else:
_SHARED_EXECUTOR.submit(fetch_sar_products): ("fetch_sar_products", time.perf_counter()), logger.info(
} "Startup preload: Meshtastic cache already loaded, deferring remote map refresh to scheduled cadence"
)
logger.info("Startup priority preload starting (%d tasks)...", len(priority_funcs))
cycle_start = time.perf_counter()
futures = {
_SHARED_EXECUTOR.submit(func): (func.__name__, time.perf_counter())
for func in priority_funcs
}
for future, (name, start) in futures.items():
remaining = _STARTUP_PRIORITY_TIMEOUT_S - (time.perf_counter() - cycle_start)
if remaining <= 0:
logger.info("Startup priority budget reached; %s will continue in background", name)
continue
try:
future.result(timeout=remaining)
_record_fetch_success("startup-priority", name, start)
except concurrent.futures.TimeoutError:
logger.info(
"Startup priority task still warming after %.1fs: %s",
time.perf_counter() - start,
name,
)
except Exception as e:
_record_fetch_failure("startup-priority", name, start, e)
logger.info("Startup preload: deferring Playwright Liveuamap scraper to scheduled cadence")
_save_intel_startup_cache()
_schedule_delayed_startup_heavy_refresh()
logger.info("Startup priority preload complete; slow synthesis is warming in background.")
return
refresh_funcs = [
fetch_airports,
update_fast_data,
update_slow_data,
fetch_volcanoes,
fetch_viirs_change_nodes,
fetch_unusual_whales,
fetch_fimi,
fetch_gdelt,
fetch_uap_sightings,
fetch_wastewater,
fetch_crowdthreat,
fetch_sar_catalog,
fetch_sar_products,
]
if not startup_mode or not meshtastic_seeded: if not startup_mode or not meshtastic_seeded:
futures[_SHARED_EXECUTOR.submit(fetch_meshtastic_nodes)] = ( refresh_funcs.append(fetch_meshtastic_nodes)
"fetch_meshtastic_nodes",
time.perf_counter(),
)
else: else:
logger.info( logger.info(
"Startup preload: Meshtastic cache already loaded, deferring remote map refresh to scheduled cadence" "Startup preload: Meshtastic cache already loaded, deferring remote map refresh to scheduled cadence"
) )
for future, (name, start) in futures.items(): if not startup_mode:
try: refresh_funcs.append(update_liveuamap)
future.result(timeout=_TASK_HARD_TIMEOUT_S) else:
duration = time.perf_counter() - start logger.info("Startup preload: deferring Playwright Liveuamap scraper to scheduled cadence")
from services.fetch_health import record_success _run_tasks("full-refresh", refresh_funcs, max_concurrency=_STARTUP_HEAVY_CONCURRENCY)
record_success(name, duration_s=duration)
if duration > _SLOW_FETCH_S:
logger.warning(f"full-refresh task slow: {name} took {duration:.2f}s")
except Exception as e:
duration = time.perf_counter() - start
from services.fetch_health import record_failure
record_failure(name, error=e, duration_s=duration)
logger.exception(f"full-refresh task failed: {name}")
# Run CCTV ingest immediately so cameras are available on first request # Run CCTV ingest immediately so cameras are available on first request
# (the scheduled job also runs every 10 min for ongoing refresh). # (the scheduled job also runs every 10 min for ongoing refresh).
if startup_mode: if startup_mode:
@@ -408,7 +632,7 @@ def update_all_data(*, startup_mode: bool = False):
_scheduler = None _scheduler = None
_STARTUP_CCTV_INGEST_DELAY_S = 30 _STARTUP_CCTV_INGEST_DELAY_S = int(os.environ.get("SHADOWBROKER_STARTUP_CCTV_INGEST_DELAY_S", "180"))
_FINANCIAL_REFRESH_MINUTES = 30 _FINANCIAL_REFRESH_MINUTES = 30
@@ -32,7 +32,7 @@ _REFRESH_INTERVAL_S = 5 * 24 * 3600
_LIST_TIMEOUT_S = 30 _LIST_TIMEOUT_S = 30
_DOWNLOAD_TIMEOUT_S = 600 _DOWNLOAD_TIMEOUT_S = 600
_USER_AGENT = ( _USER_AGENT = (
"ShadowBroker-OSINT/0.9.7 " "ShadowBroker-OSINT/0.9.75 "
"(+https://github.com/BigBodyCobain/Shadowbroker; " "(+https://github.com/BigBodyCobain/Shadowbroker; "
"contact: bigbodycobain@gmail.com)" "contact: bigbodycobain@gmail.com)"
) )
+158 -5
View File
@@ -15,7 +15,7 @@ import time
import heapq import heapq
from datetime import datetime, timedelta from datetime import datetime, timedelta
from pathlib import Path from pathlib import Path
from services.network_utils import fetch_with_curl from services.network_utils import external_curl_fallback_enabled, fetch_with_curl
from services.fetchers._store import latest_data, _data_lock, _mark_fresh from services.fetchers._store import latest_data, _data_lock, _mark_fresh
from services.fetchers.nuforc_enrichment import enrich_sighting from services.fetchers.nuforc_enrichment import enrich_sighting
from services.fetchers.retry import with_retry from services.fetchers.retry import with_retry
@@ -685,6 +685,8 @@ _NUFORC_TOKEN = os.environ.get("NUFORC_MAPBOX_TOKEN", "").strip()
_NUFORC_RADIUS_M = 200_000 # 200 km query radius _NUFORC_RADIUS_M = 200_000 # 200 km query radius
_NUFORC_LIMIT = 50 # max features per tilequery call _NUFORC_LIMIT = 50 # max features per tilequery call
_NUFORC_RECENT_DAYS = int(os.environ.get("NUFORC_RECENT_DAYS", "60")) _NUFORC_RECENT_DAYS = int(os.environ.get("NUFORC_RECENT_DAYS", "60"))
_NUFORC_HF_FALLBACK_LIMIT = max(25, int(os.environ.get("NUFORC_HF_FALLBACK_LIMIT", "250")))
_NUFORC_HF_GEOCODE_LIMIT = max(25, int(os.environ.get("NUFORC_HF_GEOCODE_LIMIT", "150")))
_NUFORC_GEOCODE_WORKERS = max(1, int(os.environ.get("NUFORC_GEOCODE_WORKERS", "1"))) _NUFORC_GEOCODE_WORKERS = max(1, int(os.environ.get("NUFORC_GEOCODE_WORKERS", "1")))
# Photon (Komoot) is more lenient than Nominatim — ~200ms per query in # Photon (Komoot) is more lenient than Nominatim — ~200ms per query in
# practice, so a 0.3s spacing keeps us well under any soft throttle while # practice, so a 0.3s spacing keeps us well under any soft throttle while
@@ -1034,6 +1036,14 @@ def _nuforc_fetch_month_live(yyyymm: str, cookie_jar: Path) -> list[dict]:
index_url = _NUFORC_LIVE_INDEX_URL.format(yyyymm=yyyymm) index_url = _NUFORC_LIVE_INDEX_URL.format(yyyymm=yyyymm)
ajax_url = _NUFORC_LIVE_AJAX_URL.format(yyyymm=yyyymm) ajax_url = _NUFORC_LIVE_AJAX_URL.format(yyyymm=yyyymm)
if not external_curl_fallback_enabled():
logger.warning(
"NUFORC live: external curl disabled on Windows for %s; "
"set SHADOWBROKER_ENABLE_WINDOWS_CURL_FALLBACK=1 to opt in.",
yyyymm,
)
return []
# Step 1: GET the month index to capture session cookies + fresh nonce. # Step 1: GET the month index to capture session cookies + fresh nonce.
try: try:
index_res = subprocess.run( index_res = subprocess.run(
@@ -1340,6 +1350,143 @@ def _build_recent_uap_sightings() -> list[dict]:
return sightings return sightings
def _split_uap_location(location: str) -> tuple[str, str, str]:
parts = [p.strip() for p in str(location or "").split(",") if p.strip()]
city = parts[0] if parts else ""
state = ""
country = ""
if len(parts) >= 2:
state = parts[1]
if len(parts) >= 3:
country = parts[-1]
if country and country.upper() in _US_COUNTRY_ALIASES:
country = "US"
return city, state, country
def _build_uap_sightings_from_hf_mirror() -> list[dict]:
"""Build visible UAP points from the public Hugging Face NUFORC mirror.
This is a resilience fallback for local/Windows runs where nuforc.org is
Cloudflare-gated and the Mapbox token is not configured. It is not as fresh
as the live NUFORC AJAX feed, but it keeps the layer visible and cached.
"""
from services.fetchers.nuforc_enrichment import _HF_CSV_URL, _parse_date
from services.geocode_validate import coord_in_country
try:
response = fetch_with_curl(_HF_CSV_URL, timeout=180, follow_redirects=True)
if not response or response.status_code != 200:
logger.warning(
"UAP sightings: HF fallback failed HTTP %s",
getattr(response, "status_code", "None"),
)
return []
except Exception as e:
logger.warning("UAP sightings: HF fallback download failed: %s", e)
return []
candidates: list[dict] = []
try:
reader = csv.DictReader(io.StringIO(response.text))
for row in reader:
occurred = _parse_date(
row.get("Occurred", "")
or row.get("Date / Time", "")
or row.get("Date", "")
)
if not occurred:
continue
raw_location = _normalize_uap_location(
row.get("Location", "")
or row.get("City", "")
or row.get("location", "")
)
if not raw_location:
continue
city, state, country = _split_uap_location(raw_location)
if not city:
continue
sighting_id = str(row.get("Sighting", "") or "").strip()
if not sighting_id:
sighting_id = hashlib.sha1(
f"{occurred}|{raw_location}|{row.get('Summary', '')}".encode("utf-8", "ignore")
).hexdigest()[:12]
summary = (row.get("Summary", "") or row.get("Text", "") or "Sighting reported").strip()
if len(summary) > 280:
summary = summary[:277] + "..."
candidates.append({
"id": f"NUFORC-{sighting_id}",
"occurred": occurred,
"posted": _parse_date(row.get("Posted", "") or row.get("Reported", "")) or occurred,
"location": raw_location,
"city": city,
"state": state,
"country": country or _uap_country_from_location(raw_location, state),
"shape_raw": (row.get("Shape", "") or "Unknown").strip(),
"duration": (row.get("Duration", "") or "").strip(),
"summary": summary,
})
except Exception as e:
logger.warning("UAP sightings: HF fallback parse failed: %s", e)
return []
candidates.sort(key=lambda row: (row["occurred"], row["posted"], row["id"]), reverse=True)
candidates = candidates[:_NUFORC_HF_FALLBACK_LIMIT]
location_cache = _load_nuforc_location_cache()
sightings: list[dict] = []
geocoded = 0
for row in candidates:
coords = location_cache.get(row["location"])
if row["location"] not in location_cache and geocoded < _NUFORC_HF_GEOCODE_LIMIT:
try:
coords = _geocode_uap_location(
row["location"], row["city"], row["state"], row["country"]
)
except Exception:
coords = None
location_cache[row["location"]] = coords
geocoded += 1
if geocoded < _NUFORC_HF_GEOCODE_LIMIT:
time.sleep(_NUFORC_GEOCODE_SPACING_S)
if not coords:
continue
if row.get("country"):
try:
inside = coord_in_country(coords[0], coords[1], row["country"])
except Exception:
inside = None
if inside is False:
continue
shape_raw = row["shape_raw"] or "Unknown"
sightings.append({
"id": row["id"],
"date_time": row["occurred"],
"city": row["city"],
"state": row["state"],
"country": row["country"],
"shape": _normalize_uap_shape(shape_raw) if shape_raw != "Unknown" else "unknown",
"shape_raw": shape_raw,
"duration": row["duration"],
"summary": row["summary"],
"posted": row["posted"],
"lat": float(coords[0]),
"lng": float(coords[1]),
"count": 1,
"source": "NUFORC-HF",
})
_save_nuforc_location_cache(location_cache)
logger.info(
"UAP sightings: %d mapped reports from HF fallback (%d candidates, %d geocoded)",
len(sightings),
len(candidates),
geocoded,
)
return sightings
@with_retry(max_retries=1, base_delay=5) @with_retry(max_retries=1, base_delay=5)
def fetch_uap_sightings(*, force_refresh: bool = False): def fetch_uap_sightings(*, force_refresh: bool = False):
"""Fetch last-year UAP sightings from NUFORC. """Fetch last-year UAP sightings from NUFORC.
@@ -1355,12 +1502,18 @@ def fetch_uap_sightings(*, force_refresh: bool = False):
sightings = _load_nuforc_sightings_cache(force_refresh=force_refresh) sightings = _load_nuforc_sightings_cache(force_refresh=force_refresh)
if sightings is None: if sightings is None:
sightings = _build_recent_uap_sightings() try:
_save_nuforc_sightings_cache(sightings) sightings = _build_recent_uap_sightings()
except Exception as e:
logger.warning("UAP sightings: live NUFORC rebuild failed, using fallback: %s", e)
sightings = _build_uap_sightings_from_hf_mirror()
if sightings:
_save_nuforc_sightings_cache(sightings)
with _data_lock: with _data_lock:
latest_data["uap_sightings"] = sightings latest_data["uap_sightings"] = sightings or []
_mark_fresh("uap_sightings") if sightings:
_mark_fresh("uap_sightings")
return return
cutoff = datetime.utcnow() - timedelta(days=_NUFORC_RECENT_DAYS) cutoff = datetime.utcnow() - timedelta(days=_NUFORC_RECENT_DAYS)
+45 -15
View File
@@ -256,7 +256,17 @@ PRIVATE_JET_TYPES = {
# Flight trails state # Flight trails state
flight_trails = {} # {icao_hex: {points: [[lat, lng, alt, ts], ...], last_seen: ts}} flight_trails = {} # {icao_hex: {points: [[lat, lng, alt, ts], ...], last_seen: ts}}
_trails_lock = threading.Lock() _trails_lock = threading.Lock()
_MAX_TRACKED_TRAILS = 2000 _MAX_TRACKED_TRAILS = 20000
def get_flight_trail(icao24: str) -> list:
"""Return the accumulated trail for a single aircraft without expanding live payloads."""
hex_id = str(icao24 or "").strip().lower()
if not hex_id:
return []
with _trails_lock:
points = flight_trails.get(hex_id, {}).get("points", [])
return [list(point) for point in points]
# Route enrichment is now served from services.fetchers.route_database, which # Route enrichment is now served from services.fetchers.route_database, which
# bulk-loads vrs-standing-data.adsb.lol/routes.csv.gz once per day and looks up # bulk-loads vrs-standing-data.adsb.lol/routes.csv.gz once per day and looks up
@@ -612,24 +622,30 @@ def _classify_and_publish(all_adsb_flights):
) )
# --- Trail Accumulation --- # --- Trail Accumulation ---
_TRAIL_INTERVAL_S = 600 # only record a new trail point every 10 minutes _TRAIL_INTERVAL_S = 60 # selected trails need enough resolution to show where unknown-route traffic came from
def _accumulate_trail(f, now_ts, check_route=True): def _accumulate_trail(f, now_ts, attach_known_route_trail=False):
hex_id = f.get("icao24", "").lower() hex_id = f.get("icao24", "").lower()
if not hex_id: if not hex_id:
return 0, None return 0, None
if check_route and f.get("origin_name", "UNKNOWN") != "UNKNOWN":
f["trail"] = [] def _known_route_name(value):
return 0, hex_id normalized = str(value or "").strip().upper()
return bool(normalized and normalized != "UNKNOWN")
has_known_route = bool(
(f.get("origin_loc") and f.get("dest_loc"))
or (_known_route_name(f.get("origin_name")) and _known_route_name(f.get("dest_name")))
)
lat, lng, alt = f.get("lat"), f.get("lng"), f.get("alt", 0) lat, lng, alt = f.get("lat"), f.get("lng"), f.get("alt", 0)
if lat is None or lng is None: if lat is None or lng is None:
f["trail"] = flight_trails.get(hex_id, {}).get("points", []) f["trail"] = [] if has_known_route and not attach_known_route_trail else flight_trails.get(hex_id, {}).get("points", [])
return 0, hex_id return 0, hex_id
point = [round(lat, 5), round(lng, 5), round(alt, 1), round(now_ts)] point = [round(lat, 5), round(lng, 5), round(alt, 1), round(now_ts)]
if hex_id not in flight_trails: if hex_id not in flight_trails:
flight_trails[hex_id] = {"points": [], "last_seen": now_ts} flight_trails[hex_id] = {"points": [], "last_seen": now_ts}
trail_data = flight_trails[hex_id] trail_data = flight_trails[hex_id]
# Only append a new point if 10 minutes have passed since the last one # Only append a new point if enough time has passed since the last one
last_point_ts = trail_data["points"][-1][3] if trail_data["points"] else 0 last_point_ts = trail_data["points"][-1][3] if trail_data["points"] else 0
if now_ts - last_point_ts < _TRAIL_INTERVAL_S: if now_ts - last_point_ts < _TRAIL_INTERVAL_S:
trail_data["last_seen"] = now_ts trail_data["last_seen"] = now_ts
@@ -644,32 +660,39 @@ def _classify_and_publish(all_adsb_flights):
trail_data["last_seen"] = now_ts trail_data["last_seen"] = now_ts
if len(trail_data["points"]) > 200: if len(trail_data["points"]) > 200:
trail_data["points"] = trail_data["points"][-200:] trail_data["points"] = trail_data["points"][-200:]
f["trail"] = trail_data["points"] # Keep known-route flights visually clean in the main payload; selected
# detail panels can still fetch this server-side trail to compute
# observed fuel/CO2 burn.
f["trail"] = [] if has_known_route and not attach_known_route_trail else trail_data["points"]
return 1, hex_id return 1, hex_id
now_ts = datetime.utcnow().timestamp() now_ts = datetime.utcnow().timestamp()
with _data_lock: with _data_lock:
commercial_snapshot = copy.deepcopy(latest_data.get("commercial_flights", []))
private_jets_snapshot = copy.deepcopy(latest_data.get("private_jets", []))
private_ga_snapshot = copy.deepcopy(latest_data.get("private_flights", []))
military_snapshot = copy.deepcopy(latest_data.get("military_flights", [])) military_snapshot = copy.deepcopy(latest_data.get("military_flights", []))
tracked_snapshot = copy.deepcopy(latest_data.get("tracked_flights", [])) tracked_snapshot = copy.deepcopy(latest_data.get("tracked_flights", []))
raw_flights_snapshot = list(latest_data.get("flights", [])) raw_flights_snapshot = list(latest_data.get("flights", []))
# Commercial/private: skip trail if route is known (route line replaces trail) # Accumulate trails for every aircraft so selected details can estimate
route_check_lists = [commercial, private_jets, private_ga] # observed fuel/CO2 burn. Known-route flights keep an empty payload trail so
# Tracked + military: ALWAYS accumulate trails (high-interest flights) # the route line, not historical breadcrumbs, remains the visible map path.
always_trail_lists = [existing_tracked, military_snapshot] route_check_lists = [commercial_snapshot, private_jets_snapshot, private_ga_snapshot]
always_trail_lists = [tracked_snapshot, military_snapshot]
seen_hexes = set() seen_hexes = set()
trail_count = 0 trail_count = 0
with _trails_lock: with _trails_lock:
for flist in route_check_lists: for flist in route_check_lists:
for f in flist: for f in flist:
count, hex_id = _accumulate_trail(f, now_ts, check_route=True) count, hex_id = _accumulate_trail(f, now_ts, attach_known_route_trail=False)
trail_count += count trail_count += count
if hex_id: if hex_id:
seen_hexes.add(hex_id) seen_hexes.add(hex_id)
for flist in always_trail_lists: for flist in always_trail_lists:
for f in flist: for f in flist:
count, hex_id = _accumulate_trail(f, now_ts, check_route=False) count, hex_id = _accumulate_trail(f, now_ts, attach_known_route_trail=False)
trail_count += count trail_count += count
if hex_id: if hex_id:
seen_hexes.add(hex_id) seen_hexes.add(hex_id)
@@ -693,6 +716,13 @@ def _classify_and_publish(all_adsb_flights):
f"Trail accumulation: {trail_count} active trails, {len(stale_keys)} pruned, {len(flight_trails)} total" f"Trail accumulation: {trail_count} active trails, {len(stale_keys)} pruned, {len(flight_trails)} total"
) )
with _data_lock:
latest_data["commercial_flights"] = commercial_snapshot
latest_data["private_jets"] = private_jets_snapshot
latest_data["private_flights"] = private_ga_snapshot
latest_data["tracked_flights"] = tracked_snapshot
latest_data["military_flights"] = military_snapshot
# --- GPS Jamming Detection --- # --- GPS Jamming Detection ---
# Uses NACp (Navigation Accuracy Category Position) from ADS-B to infer # Uses NACp (Navigation Accuracy Category Position) from ADS-B to infer
# GPS interference zones, similar to GPSJam.org / Flightradar24. # GPS interference zones, similar to GPSJam.org / Flightradar24.
+24
View File
@@ -15,6 +15,24 @@ from services.fetchers.retry import with_retry
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
def _env_flag(name: str) -> str:
return str(os.getenv(name, "")).strip().lower()
def liveuamap_scraper_enabled() -> bool:
"""Return whether the Playwright-based LiveUAMap scraper should run.
It is useful enrichment, but it starts a browser/Node driver and must not be
allowed to destabilize Windows local startup.
"""
setting = _env_flag("SHADOWBROKER_ENABLE_LIVEUAMAP_SCRAPER")
if setting in {"1", "true", "yes", "on"}:
return True
if setting in {"0", "false", "no", "off"}:
return False
return os.name != "nt"
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Ships (AIS + Carriers) # Ships (AIS + Carriers)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -191,6 +209,12 @@ def update_liveuamap():
if not is_any_active("global_incidents"): if not is_any_active("global_incidents"):
return return
if not liveuamap_scraper_enabled():
logger.info(
"Liveuamap scraper disabled for this runtime; set "
"SHADOWBROKER_ENABLE_LIVEUAMAP_SCRAPER=1 to opt in."
)
return
logger.info("Running scheduled Liveuamap scraper...") logger.info("Running scheduled Liveuamap scraper...")
try: try:
from services.liveuamap_scraper import fetch_liveuamap from services.liveuamap_scraper import fetch_liveuamap
+1 -1
View File
@@ -182,7 +182,7 @@ def fetch_meshtastic_nodes():
callsign = str(getattr(get_settings(), "MESHTASTIC_OPERATOR_CALLSIGN", "") or "").strip() callsign = str(getattr(get_settings(), "MESHTASTIC_OPERATOR_CALLSIGN", "") or "").strip()
except Exception: except Exception:
callsign = "" callsign = ""
ua_base = "ShadowBroker-OSINT/0.9.7 (+https://github.com/BigBodyCobain/Shadowbroker; contact: bigbodycobain@gmail.com; 24h polling)" ua_base = "ShadowBroker-OSINT/0.9.75 (+https://github.com/BigBodyCobain/Shadowbroker; contact: bigbodycobain@gmail.com; 24h polling)"
user_agent = f"{ua_base}; node={callsign}" if callsign else ua_base user_agent = f"{ua_base}; node={callsign}" if callsign else ua_base
try: try:
+8
View File
@@ -6,6 +6,7 @@ import time
import requests import requests
from services.network_utils import fetch_with_curl from services.network_utils import fetch_with_curl
from services.fetchers._store import latest_data, _data_lock, _mark_fresh from services.fetchers._store import latest_data, _data_lock, _mark_fresh
from services.fetchers.emissions import get_emissions_info
from services.fetchers.plane_alert import enrich_with_plane_alert from services.fetchers.plane_alert import enrich_with_plane_alert
logger = logging.getLogger("services.data_fetcher") logger = logging.getLogger("services.data_fetcher")
@@ -289,6 +290,13 @@ def fetch_military_flights():
remaining_mil = [] remaining_mil = []
for mf in military_flights: for mf in military_flights:
enrich_with_plane_alert(mf) enrich_with_plane_alert(mf)
model = mf.get("model")
if not model or str(model).strip().lower() in {"", "unknown"}:
model = mf.get("alert_type") or ""
if model:
emissions = get_emissions_info(model)
if emissions:
mf["emissions"] = emissions
if mf.get("alert_category"): if mf.get("alert_category"):
mf["type"] = "tracked_flight" mf["type"] = "tracked_flight"
tracked_mil.append(mf) tracked_mil.append(mf)
+1 -1
View File
@@ -25,7 +25,7 @@ _REFRESH_INTERVAL_S = 5 * 24 * 3600
_HTTP_TIMEOUT_S = 60 _HTTP_TIMEOUT_S = 60
_USER_AGENT = ( _USER_AGENT = (
"ShadowBroker-OSINT/0.9.7 " "ShadowBroker-OSINT/0.9.75 "
"(+https://github.com/BigBodyCobain/Shadowbroker; " "(+https://github.com/BigBodyCobain/Shadowbroker; "
"contact: bigbodycobain@gmail.com)" "contact: bigbodycobain@gmail.com)"
) )
+124 -8
View File
@@ -15,6 +15,7 @@ Analysis features (derived from cached TLEs — no extra network requests):
import math import math
import time import time
import json import json
import os
import re import re
import logging import logging
import requests import requests
@@ -41,6 +42,38 @@ def _gmst(jd_ut1):
# CelesTrak fair use: fetch at most once per 24 hours (86400s). # CelesTrak fair use: fetch at most once per 24 hours (86400s).
# SGP4 propagation runs every 60s using cached TLEs — positions stay live. # SGP4 propagation runs every 60s using cached TLEs — positions stay live.
_CELESTRAK_FETCH_INTERVAL = 86400 # 24 hours _CELESTRAK_FETCH_INTERVAL = 86400 # 24 hours
_MIN_VISIBLE_SATELLITE_CATALOG = int(os.environ.get("SHADOWBROKER_MIN_VISIBLE_SATELLITES", "350"))
_MAX_VISIBLE_SATELLITE_CATALOG = int(os.environ.get("SHADOWBROKER_MAX_VISIBLE_SATELLITES", "450"))
_CELESTRAK_VISIBLE_GROUPS = {
"military": {"mission": "military", "sat_type": "Military / Defense"},
"radar": {"mission": "sar", "sat_type": "Radar / SAR"},
"resource": {"mission": "earth_observation", "sat_type": "Earth Observation"},
"weather": {"mission": "weather", "sat_type": "Weather / Meteorology"},
"gnss": {"mission": "navigation", "sat_type": "GNSS / Navigation"},
"science": {"mission": "science", "sat_type": "Science"},
}
_TLE_VISIBLE_FALLBACK_TERMS = {
"COSMOS": {"mission": "military", "sat_type": "Russian / Soviet Military"},
"USA": {"mission": "military", "sat_type": "US Military / NRO"},
"NROL": {"mission": "military", "sat_type": "Classified NRO"},
"GPS": {"mission": "navigation", "sat_type": "GPS Navigation"},
"GALILEO": {"mission": "navigation", "sat_type": "Galileo Navigation"},
"BEIDOU": {"mission": "navigation", "sat_type": "BeiDou Navigation"},
"GLONASS": {"mission": "navigation", "sat_type": "GLONASS Navigation"},
"NOAA": {"mission": "weather", "sat_type": "NOAA Weather"},
"METEOR": {"mission": "weather", "sat_type": "Meteor Weather"},
"SENTINEL": {"mission": "earth_observation", "sat_type": "Sentinel Earth Observation"},
"LANDSAT": {"mission": "earth_observation", "sat_type": "Landsat Earth Observation"},
"WORLDVIEW": {"mission": "commercial_imaging", "sat_type": "Maxar High-Res"},
"PLEIADES": {"mission": "commercial_imaging", "sat_type": "Airbus Imaging"},
"SKYSAT": {"mission": "commercial_imaging", "sat_type": "Planet Video"},
"JILIN": {"mission": "commercial_imaging", "sat_type": "Jilin Imaging"},
"FLOCK": {"mission": "commercial_imaging", "sat_type": "PlanetScope"},
"LEMUR": {"mission": "commercial_rf", "sat_type": "Spire RF / AIS"},
"ICEYE": {"mission": "sar", "sat_type": "ICEYE SAR"},
"UMBRA": {"mission": "sar", "sat_type": "Umbra SAR"},
"CAPELLA": {"mission": "sar", "sat_type": "Capella SAR"},
}
_sat_gp_cache = {"data": None, "last_fetch": 0, "source": "none", "last_modified": None} _sat_gp_cache = {"data": None, "last_fetch": 0, "source": "none", "last_modified": None}
_sat_classified_cache = {"data": None, "gp_fetch_ts": 0} _sat_classified_cache = {"data": None, "gp_fetch_ts": 0}
_SAT_CACHE_PATH = Path(__file__).parent.parent.parent / "data" / "sat_gp_cache.json" _SAT_CACHE_PATH = Path(__file__).parent.parent.parent / "data" / "sat_gp_cache.json"
@@ -564,9 +597,61 @@ def _parse_tle_to_gp(name, norad_id, line1, line2):
return None return None
def _annotate_celestrak_group(records: list[dict], group: str) -> list[dict]:
meta = _CELESTRAK_VISIBLE_GROUPS.get(group, {})
out = []
for sat in records:
if not isinstance(sat, dict):
continue
item = dict(sat)
item["_SB_GROUP"] = group
if meta:
item["_SB_GROUP_META"] = meta
out.append(item)
return out
def _fetch_visible_celestrak_catalog(headers: dict | None = None) -> list[dict]:
"""Fetch bounded CelesTrak groups used by the visible satellite layer.
The full ``active`` catalog is too large and frequently times out on local
startup. These groups cover the visible operational set users expect
without pulling Starlink-scale constellations into the map.
"""
headers = headers or {}
merged: dict[int, dict] = {}
for group in _CELESTRAK_VISIBLE_GROUPS:
url = f"https://celestrak.org/NORAD/elements/gp.php?GROUP={group}&FORMAT=json"
try:
response = fetch_with_curl(url, timeout=15, headers=headers)
if response.status_code != 200:
logger.debug("Satellites: CelesTrak group %s returned HTTP %s", group, response.status_code)
continue
gp_data = response.json()
if not isinstance(gp_data, list):
continue
for sat in _annotate_celestrak_group(gp_data, group):
norad_id = sat.get("NORAD_CAT_ID")
if norad_id is None:
continue
merged[int(norad_id)] = sat
time.sleep(0.35)
except (
requests.RequestException,
ConnectionError,
TimeoutError,
ValueError,
KeyError,
json.JSONDecodeError,
OSError,
) as e:
logger.warning("Satellites: Failed to fetch CelesTrak group %s: %s", group, e)
return list(merged.values())
def _fetch_satellites_from_tle_api(): def _fetch_satellites_from_tle_api():
"""Fallback: fetch satellite TLEs from tle.ivanstanojevic.me when CelesTrak is blocked.""" """Fallback: fetch satellite TLEs from tle.ivanstanojevic.me when CelesTrak is blocked."""
search_terms = set() search_terms = set(_TLE_VISIBLE_FALLBACK_TERMS)
for key, _ in _SAT_INTEL_DB: for key, _ in _SAT_INTEL_DB:
term = key.split()[0] if len(key.split()) > 1 and key.split()[0] in ("USA", "NROL") else key term = key.split()[0] if len(key.split()) > 1 and key.split()[0] in ("USA", "NROL") else key
search_terms.add(term) search_terms.add(term)
@@ -591,8 +676,13 @@ def _fetch_satellites_from_tle_api():
sat_id = gp.get("NORAD_CAT_ID") sat_id = gp.get("NORAD_CAT_ID")
if sat_id not in seen_ids: if sat_id not in seen_ids:
seen_ids.add(sat_id) seen_ids.add(sat_id)
if term in _TLE_VISIBLE_FALLBACK_TERMS:
gp["_SB_GROUP"] = f"tle:{term}"
gp["_SB_GROUP_META"] = _TLE_VISIBLE_FALLBACK_TERMS[term]
all_results.append(gp) all_results.append(gp)
time.sleep(1) # Polite delay between requests if len(all_results) >= _MAX_VISIBLE_SATELLITE_CATALOG:
return all_results
time.sleep(0.15) # Polite delay between requests
except ( except (
requests.RequestException, requests.RequestException,
ConnectionError, ConnectionError,
@@ -644,18 +734,34 @@ def fetch_satellites():
if ( if (
_sat_gp_cache["data"] is None _sat_gp_cache["data"] is None
or len(_sat_gp_cache.get("data") or []) < _MIN_VISIBLE_SATELLITE_CATALOG
or (now_ts - _sat_gp_cache["last_fetch"]) > _CELESTRAK_FETCH_INTERVAL or (now_ts - _sat_gp_cache["last_fetch"]) > _CELESTRAK_FETCH_INTERVAL
): ):
gp_urls = [
"https://celestrak.org/NORAD/elements/gp.php?GROUP=active&FORMAT=json",
"https://celestrak.com/NORAD/elements/gp.php?GROUP=active&FORMAT=json",
]
# Build conditional request headers (CelesTrak fair use) # Build conditional request headers (CelesTrak fair use)
headers = {} headers = {}
if _sat_gp_cache.get("last_modified"): if _sat_gp_cache.get("last_modified"):
headers["If-Modified-Since"] = _sat_gp_cache["last_modified"] headers["If-Modified-Since"] = _sat_gp_cache["last_modified"]
visible_data = _fetch_visible_celestrak_catalog(headers=headers)
if len(visible_data) >= _MIN_VISIBLE_SATELLITE_CATALOG:
_sat_gp_cache["data"] = visible_data
_sat_gp_cache["last_fetch"] = now_ts
_sat_gp_cache["source"] = "celestrak_visible_groups"
_save_sat_cache(visible_data)
_snapshot_current_tles(visible_data)
logger.info(
"Satellites: Downloaded %d GP records from visible CelesTrak groups",
len(visible_data),
)
gp_urls = [
"https://celestrak.org/NORAD/elements/gp.php?GROUP=active&FORMAT=json",
"https://celestrak.com/NORAD/elements/gp.php?GROUP=active&FORMAT=json",
]
for url in gp_urls: for url in gp_urls:
if len(_sat_gp_cache.get("data") or []) >= _MIN_VISIBLE_SATELLITE_CATALOG:
break
try: try:
response = fetch_with_curl(url, timeout=15, headers=headers) response = fetch_with_curl(url, timeout=15, headers=headers)
if response.status_code == 304: if response.status_code == 304:
@@ -696,7 +802,10 @@ def fetch_satellites():
logger.warning(f"Satellites: Failed to fetch from {url}: {e}") logger.warning(f"Satellites: Failed to fetch from {url}: {e}")
continue continue
if _sat_gp_cache["data"] is None: if (
_sat_gp_cache["data"] is None
or len(_sat_gp_cache.get("data") or []) < _MIN_VISIBLE_SATELLITE_CATALOG
):
logger.info("Satellites: CelesTrak unreachable, trying TLE fallback API...") logger.info("Satellites: CelesTrak unreachable, trying TLE fallback API...")
try: try:
fallback_data = _fetch_satellites_from_tle_api() fallback_data = _fetch_satellites_from_tle_api()
@@ -757,6 +866,9 @@ def fetch_satellites():
owner = sat.get("OWNER", sat.get("OBJECT_OWNER", "")) owner = sat.get("OWNER", sat.get("OBJECT_OWNER", ""))
if owner in _OWNER_CODE_MAP: if owner in _OWNER_CODE_MAP:
intel = {"country": _OWNER_CODE_MAP[owner], "mission": "general", "sat_type": "Unclassified"} intel = {"country": _OWNER_CODE_MAP[owner], "mission": "general", "sat_type": "Unclassified"}
if not intel and sat.get("_SB_GROUP_META"):
intel = dict(sat["_SB_GROUP_META"])
intel.setdefault("country", "Unknown")
if not intel: if not intel:
continue continue
@@ -818,7 +930,11 @@ def fetch_satellites():
now.year, now.month, now.day, now.hour, now.minute, now.second + now.microsecond / 1e6 now.year, now.month, now.day, now.hour, now.minute, now.second + now.microsecond / 1e6
) )
for s in all_sats: for source_sat in all_sats:
# Keep the classified cache immutable. The render payload below
# strips orbital fields after propagation, and mutating the cached
# entry would make the next refresh unable to position satellites.
s = dict(source_sat)
try: try:
mean_motion = s.get("MEAN_MOTION") mean_motion = s.get("MEAN_MOTION")
ecc = s.get("ECCENTRICITY") ecc = s.get("ECCENTRICITY")
+12 -15
View File
@@ -390,15 +390,9 @@ class MeshtasticTransport:
def _mqtt_config() -> tuple[str, int, str, str]: def _mqtt_config() -> tuple[str, int, str, str]:
"""Return (broker, port, user, password) from settings.""" """Return (broker, port, user, password) from settings."""
try: try:
from services.config import get_settings from services.meshtastic_mqtt_settings import mqtt_connection_config
s = get_settings() return mqtt_connection_config()
return (
str(s.MESH_MQTT_BROKER or "mqtt.meshtastic.org"),
int(s.MESH_MQTT_PORT or 1883),
str(s.MESH_MQTT_USER or "meshdev"),
str(s.MESH_MQTT_PASS or "large4cats"),
)
except Exception: except Exception:
return ("mqtt.meshtastic.org", 1883, "meshdev", "large4cats") return ("mqtt.meshtastic.org", 1883, "meshdev", "large4cats")
@@ -433,8 +427,9 @@ class MeshtasticTransport:
def _resolve_psk(cls) -> bytes: def _resolve_psk(cls) -> bytes:
"""Return the PSK from config, or the default LongFast key if empty.""" """Return the PSK from config, or the default LongFast key if empty."""
try: try:
from services.config import get_settings from services.meshtastic_mqtt_settings import mqtt_psk_hex
raw = str(getattr(get_settings(), "MESH_MQTT_PSK", "") or "").strip()
raw = mqtt_psk_hex()
except Exception: except Exception:
raw = "" raw = ""
if not raw: if not raw:
@@ -449,7 +444,10 @@ class MeshtasticTransport:
@staticmethod @staticmethod
def mesh_address_for_sender(sender_id: str) -> str: def mesh_address_for_sender(sender_id: str) -> str:
"""Return the synthetic public mesh address used for MQTT-originated sends.""" """Return the public mesh address used for MQTT-originated sends."""
parsed = MeshtasticTransport._parse_node_id(sender_id)
if parsed is not None:
return f"!{parsed:08x}"
return f"!{MeshtasticTransport._stable_node_id(sender_id):08x}" return f"!{MeshtasticTransport._stable_node_id(sender_id):08x}"
@staticmethod @staticmethod
@@ -489,7 +487,8 @@ class MeshtasticTransport:
# Generate IDs # Generate IDs
packet_id = random.randint(1, 0xFFFFFFFF) packet_id = random.randint(1, 0xFFFFFFFF)
from_node = self._stable_node_id(envelope.sender_id) parsed_sender = self._parse_node_id(envelope.sender_id)
from_node = parsed_sender if parsed_sender is not None else self._stable_node_id(envelope.sender_id)
direct_node = self._parse_node_id(envelope.destination) direct_node = self._parse_node_id(envelope.destination)
to_node = direct_node if direct_node is not None else 0xFFFFFFFF to_node = direct_node if direct_node is not None else 0xFFFFFFFF
@@ -529,9 +528,7 @@ class MeshtasticTransport:
error_msg[0] = f"MQTT connect refused: rc={rc}" error_msg[0] = f"MQTT connect refused: rc={rc}"
client.disconnect() client.disconnect()
client = mqtt.Client( client = mqtt.Client(client_id=f"meshchat-tx-{envelope.message_id[:8]}", protocol=mqtt.MQTTv311)
client_id=f"shadowbroker-tx-{envelope.message_id[:8]}", protocol=mqtt.MQTTv311
)
broker, port, user, pw = self._mqtt_config() broker, port, user, pw = self._mqtt_config()
client.username_pw_set(user, pw) client.username_pw_set(user, pw)
client.on_connect = _on_connect client.on_connect = _on_connect
+34 -1
View File
@@ -230,11 +230,16 @@ def _raw_fallback_allowed() -> bool:
return False return False
def _generated_secret_file() -> Path:
return DATA_DIR / "secure_storage_secret.key"
def _get_storage_secret() -> str | None: def _get_storage_secret() -> str | None:
"""Return the operator-supplied secure storage secret, or None.""" """Return the operator-supplied or local generated secure storage secret."""
secret = os.environ.get("MESH_SECURE_STORAGE_SECRET", "").strip() secret = os.environ.get("MESH_SECURE_STORAGE_SECRET", "").strip()
if secret: if secret:
return secret return secret
secret_file_override = os.environ.get("MESH_SECURE_STORAGE_SECRET_FILE", "").strip()
try: try:
from services.config import get_settings from services.config import get_settings
@@ -242,8 +247,36 @@ def _get_storage_secret() -> str | None:
secret = str(getattr(settings, "MESH_SECURE_STORAGE_SECRET", "") or "").strip() secret = str(getattr(settings, "MESH_SECURE_STORAGE_SECRET", "") or "").strip()
if secret: if secret:
return secret return secret
secret_file_override = (
secret_file_override
or str(getattr(settings, "MESH_SECURE_STORAGE_SECRET_FILE", "") or "").strip()
)
except Exception: except Exception:
pass pass
if not _is_windows():
if _raw_fallback_allowed():
return None
secret_file = Path(secret_file_override or _generated_secret_file())
try:
if secret_file.exists():
secret = secret_file.read_text(encoding="utf-8").strip()
if secret:
return secret
secret_file.parent.mkdir(parents=True, exist_ok=True)
secret = _b64(os.urandom(48))
_atomic_write_text(secret_file, secret + "\n", encoding="utf-8")
try:
os.chmod(secret_file, 0o600)
except OSError:
pass
logger.info("Generated local secure storage secret at %s", secret_file)
return secret
except Exception as exc:
logger.warning(
"Failed to load or generate local secure storage secret at %s: %s",
secret_file,
exc,
)
return None return None
+7 -2
View File
@@ -8,6 +8,7 @@ from typing import Iterable
# Default subscription roots — US-only to avoid flooding the public broker. # Default subscription roots — US-only to avoid flooding the public broker.
# Users can opt into additional regions via MESH_MQTT_EXTRA_ROOTS. # Users can opt into additional regions via MESH_MQTT_EXTRA_ROOTS.
DEFAULT_ROOTS: tuple[str, ...] = ("US",) DEFAULT_ROOTS: tuple[str, ...] = ("US",)
DEFAULT_CHANNEL = "LongFast"
# Every known official region root (for UI dropdowns / manual opt-in). # Every known official region root (for UI dropdowns / manual opt-in).
ALL_OFFICIAL_ROOTS: tuple[str, ...] = ( ALL_OFFICIAL_ROOTS: tuple[str, ...] = (
@@ -107,6 +108,10 @@ def normalize_topic_filter(value: str) -> str | None:
return "/".join(parts) return "/".join(parts)
def _default_topic_for_root(root: str) -> str:
return f"msh/{root}/2/e/{DEFAULT_CHANNEL}/#"
def build_subscription_topics( def build_subscription_topics(
extra_roots: str = "", extra_roots: str = "",
extra_topics: str = "", extra_topics: str = "",
@@ -119,7 +124,7 @@ def build_subscription_topics(
# via MESH_MQTT_EXTRA_ROOTS to avoid flooding the public broker. # via MESH_MQTT_EXTRA_ROOTS to avoid flooding the public broker.
roots.extend(root for root in (normalize_root(item) for item in _split_config_values(extra_roots)) if root) roots.extend(root for root in (normalize_root(item) for item in _split_config_values(extra_roots)) if root)
topics = [f"msh/{root}/#" for root in _dedupe(roots)] topics = [_default_topic_for_root(root) for root in _dedupe(roots)]
topics.extend( topics.extend(
topic topic
for topic in ( for topic in (
@@ -137,7 +142,7 @@ def known_roots(extra_roots: str = "", include_defaults: bool = True) -> list[st
for topic in topics: for topic in topics:
if not topic.startswith("msh/") or not topic.endswith("/#"): if not topic.startswith("msh/") or not topic.endswith("/#"):
continue continue
root = normalize_root(topic[4:-2]) root = normalize_root(parse_topic_metadata(topic)["root"])
if root: if root:
roots.append(root) roots.append(root)
return _dedupe(roots) return _dedupe(roots)
@@ -0,0 +1,172 @@
from __future__ import annotations
import json
import os
import time
from pathlib import Path
from typing import Any
from services.config import get_settings
PUBLIC_DEFAULT_USER = "meshdev"
PUBLIC_DEFAULT_PASS = "large4cats"
DATA_DIR = Path(os.environ.get("SB_DATA_DIR", str(Path(__file__).parent.parent / "data")))
if not DATA_DIR.is_absolute():
DATA_DIR = Path(__file__).parent.parent / DATA_DIR
SETTINGS_FILE = DATA_DIR / "meshtastic_mqtt.json"
_cache: dict[str, Any] | None = None
_cache_ts: float = 0.0
_CACHE_TTL = 2.0
def _settings_defaults() -> dict[str, Any]:
try:
s = get_settings()
return {
"enabled": bool(getattr(s, "MESH_MQTT_ENABLED", False)),
"broker": str(getattr(s, "MESH_MQTT_BROKER", "") or "mqtt.meshtastic.org"),
"port": int(getattr(s, "MESH_MQTT_PORT", 1883) or 1883),
"username": str(getattr(s, "MESH_MQTT_USER", "") or PUBLIC_DEFAULT_USER),
"password": str(getattr(s, "MESH_MQTT_PASS", "") or PUBLIC_DEFAULT_PASS),
"psk": str(getattr(s, "MESH_MQTT_PSK", "") or ""),
"include_default_roots": bool(getattr(s, "MESH_MQTT_INCLUDE_DEFAULT_ROOTS", True)),
"extra_roots": str(getattr(s, "MESH_MQTT_EXTRA_ROOTS", "") or ""),
"extra_topics": str(getattr(s, "MESH_MQTT_EXTRA_TOPICS", "") or ""),
}
except Exception:
return {
"enabled": False,
"broker": "mqtt.meshtastic.org",
"port": 1883,
"username": PUBLIC_DEFAULT_USER,
"password": PUBLIC_DEFAULT_PASS,
"psk": "",
"include_default_roots": True,
"extra_roots": "",
"extra_topics": "",
}
def _safe_int(value: Any, default: int) -> int:
try:
parsed = int(value)
except (TypeError, ValueError):
return default
if parsed < 1 or parsed > 65535:
return default
return parsed
def _normalize(data: dict[str, Any]) -> dict[str, Any]:
defaults = _settings_defaults()
return {
"enabled": bool(data.get("enabled", defaults["enabled"])),
"broker": str(data.get("broker", defaults["broker"]) or defaults["broker"]).strip(),
"port": _safe_int(data.get("port", defaults["port"]), defaults["port"]),
"username": str(data.get("username", defaults["username"]) or "").strip(),
"password": str(data.get("password", defaults["password"]) or ""),
"psk": str(data.get("psk", defaults["psk"]) or "").strip(),
"include_default_roots": bool(data.get("include_default_roots", defaults["include_default_roots"])),
"extra_roots": str(data.get("extra_roots", defaults["extra_roots"]) or "").strip(),
"extra_topics": str(data.get("extra_topics", defaults["extra_topics"]) or "").strip(),
"updated_at": _safe_int(data.get("updated_at", 0), 0),
}
def read_meshtastic_mqtt_settings() -> dict[str, Any]:
global _cache, _cache_ts
now = time.monotonic()
if _cache is not None and (now - _cache_ts) < _CACHE_TTL:
return dict(_cache)
if not SETTINGS_FILE.exists():
result = {**_settings_defaults(), "updated_at": 0}
else:
try:
loaded = json.loads(SETTINGS_FILE.read_text(encoding="utf-8"))
except Exception:
loaded = {}
result = _normalize(loaded if isinstance(loaded, dict) else {})
_cache = result
_cache_ts = now
return dict(result)
def write_meshtastic_mqtt_settings(**updates: Any) -> dict[str, Any]:
DATA_DIR.mkdir(parents=True, exist_ok=True)
existing = read_meshtastic_mqtt_settings()
next_data = dict(existing)
for key in (
"enabled",
"broker",
"port",
"username",
"password",
"psk",
"include_default_roots",
"extra_roots",
"extra_topics",
):
if key in updates and updates[key] is not None:
next_data[key] = updates[key]
if "username" in updates and not str(updates.get("username") or "").strip() and "password" not in updates:
next_data["password"] = PUBLIC_DEFAULT_PASS
next_data["updated_at"] = int(time.time())
normalized = _normalize(next_data)
SETTINGS_FILE.write_text(json.dumps(normalized, indent=2), encoding="utf-8")
if os.name != "nt":
os.chmod(SETTINGS_FILE, 0o600)
global _cache, _cache_ts
_cache = normalized
_cache_ts = time.monotonic()
return dict(normalized)
def redacted_meshtastic_mqtt_settings(data: dict[str, Any] | None = None) -> dict[str, Any]:
source = read_meshtastic_mqtt_settings() if data is None else dict(data)
username = str(source.get("username", "") or "")
uses_default_credentials = username in ("", PUBLIC_DEFAULT_USER) and str(source.get("password", "") or "") in (
"",
PUBLIC_DEFAULT_PASS,
)
return {
"enabled": bool(source.get("enabled")),
"broker": str(source.get("broker", "")),
"port": int(source.get("port", 1883) or 1883),
"username": "" if uses_default_credentials else username,
"uses_default_credentials": uses_default_credentials,
"has_password": bool(str(source.get("password", "") or "")),
"has_psk": bool(str(source.get("psk", "") or "")),
"include_default_roots": bool(source.get("include_default_roots", True)),
"extra_roots": str(source.get("extra_roots", "") or ""),
"extra_topics": str(source.get("extra_topics", "") or ""),
"updated_at": int(source.get("updated_at", 0) or 0),
}
def mqtt_connection_config() -> tuple[str, int, str, str]:
data = read_meshtastic_mqtt_settings()
return (
str(data.get("broker") or "mqtt.meshtastic.org"),
int(data.get("port") or 1883),
str(data.get("username") or PUBLIC_DEFAULT_USER),
str(data.get("password") or PUBLIC_DEFAULT_PASS),
)
def mqtt_bridge_enabled() -> bool:
return bool(read_meshtastic_mqtt_settings().get("enabled"))
def mqtt_psk_hex() -> str:
return str(read_meshtastic_mqtt_settings().get("psk", "") or "").strip()
def mqtt_subscription_settings() -> tuple[str, str, bool]:
data = read_meshtastic_mqtt_settings()
return (
str(data.get("extra_roots", "") or ""),
str(data.get("extra_topics", "") or ""),
bool(data.get("include_default_roots", True)),
)
+35 -6
View File
@@ -1,5 +1,6 @@
import logging import logging
import json import json
import os
import subprocess import subprocess
import shutil import shutil
import time import time
@@ -20,7 +21,6 @@ _session.mount("http://", HTTPAdapter(max_retries=_retry, pool_maxsize=10))
# Find bash for curl fallback — Git bash's curl has the TLS features # Find bash for curl fallback — Git bash's curl has the TLS features
# needed to pass CDN fingerprint checks (brotli, zstd, libpsl) # needed to pass CDN fingerprint checks (brotli, zstd, libpsl)
_BASH_PATH = shutil.which("bash") or "bash"
# Cache domains where requests fails — skip straight to curl for 5 minutes # Cache domains where requests fails — skip straight to curl for 5 minutes
_domain_fail_cache: dict[str, float] = {} _domain_fail_cache: dict[str, float] = {}
@@ -39,6 +39,17 @@ class UpstreamCircuitBreakerError(OSError):
"""Raised when a domain recently failed hard and is temporarily skipped.""" """Raised when a domain recently failed hard and is temporarily skipped."""
def _env_truthy(name: str) -> bool:
return str(os.getenv(name, "")).strip().lower() in {"1", "true", "yes", "on"}
def external_curl_fallback_enabled() -> bool:
"""Return whether the backend may spawn an external curl process."""
if os.name != "nt":
return True
return _env_truthy("SHADOWBROKER_ENABLE_WINDOWS_CURL_FALLBACK")
class _DummyResponse: class _DummyResponse:
"""Minimal response object matching requests.Response interface.""" """Minimal response object matching requests.Response interface."""
def __init__(self, status_code, text): def __init__(self, status_code, text):
@@ -62,7 +73,7 @@ def fetch_with_curl(url, method="GET", json_data=None, timeout=15, headers=None,
both Python requests and the barebones Windows system curl. both Python requests and the barebones Windows system curl.
""" """
default_headers = { default_headers = {
"User-Agent": "ShadowBroker-OSINT/0.9.7 (+https://github.com/BigBodyCobain/Shadowbroker; contact: bigbodycobain@gmail.com)", "User-Agent": "ShadowBroker-OSINT/0.9.75 (+https://github.com/BigBodyCobain/Shadowbroker; contact: bigbodycobain@gmail.com)",
} }
if headers: if headers:
default_headers.update(headers) default_headers.update(headers)
@@ -98,11 +109,22 @@ def fetch_with_curl(url, method="GET", json_data=None, timeout=15, headers=None,
_circuit_breaker.pop(domain, None) _circuit_breaker.pop(domain, None)
return res return res
except (requests.RequestException, ConnectionError, TimeoutError, OSError) as e: except (requests.RequestException, ConnectionError, TimeoutError, OSError) as e:
logger.warning(f"Python requests failed for {url} ({e}), falling back to bash curl...") fallback = "falling back to curl" if external_curl_fallback_enabled() else "skipping external curl"
logger.warning(f"Python requests failed for {url} ({e}), {fallback}...")
with _cb_lock: with _cb_lock:
_domain_fail_cache[domain] = time.time() _domain_fail_cache[domain] = time.time()
# Curl fallback — reached from both _skip_requests and requests-exception paths # Curl fallback — reached from both _skip_requests and requests-exception paths
if not external_curl_fallback_enabled():
logger.warning(
"External curl fallback disabled on Windows for %s; set "
"SHADOWBROKER_ENABLE_WINDOWS_CURL_FALLBACK=1 to opt in.",
domain,
)
with _cb_lock:
_circuit_breaker[domain] = time.time()
return _DummyResponse(500, "")
_CURL_PATH = shutil.which("curl") or "curl" _CURL_PATH = shutil.which("curl") or "curl"
cmd = [_CURL_PATH, "-s", "-w", "\n%{http_code}"] cmd = [_CURL_PATH, "-s", "-w", "\n%{http_code}"]
if follow_redirects: if follow_redirects:
@@ -116,9 +138,16 @@ def fetch_with_curl(url, method="GET", json_data=None, timeout=15, headers=None,
try: try:
stdin_data = json.dumps(json_data) if (method == "POST" and json_data) else None stdin_data = json.dumps(json_data) if (method == "POST" and json_data) else None
creationflags = 0
if os.name == "nt":
creationflags = (
getattr(subprocess, "CREATE_NO_WINDOW", 0)
| getattr(subprocess, "CREATE_NEW_PROCESS_GROUP", 0)
)
res = subprocess.run( res = subprocess.run(
cmd, capture_output=True, text=True, timeout=timeout + 5, cmd, capture_output=True, text=True, timeout=timeout + 5,
input=stdin_data, encoding="utf-8", errors="replace" input=stdin_data, encoding="utf-8", errors="replace",
creationflags=creationflags,
) )
if res.returncode == 0 and (res.stdout or "").strip(): if res.returncode == 0 and (res.stdout or "").strip():
# Parse HTTP status code from -w output (last line) # Parse HTTP status code from -w output (last line)
@@ -130,12 +159,12 @@ def fetch_with_curl(url, method="GET", json_data=None, timeout=15, headers=None,
_circuit_breaker.pop(domain, None) # Clear circuit breaker on success _circuit_breaker.pop(domain, None) # Clear circuit breaker on success
return _DummyResponse(http_code, body) return _DummyResponse(http_code, body)
else: else:
logger.error(f"bash curl fallback failed: exit={res.returncode} stderr={res.stderr[:200]}") logger.error(f"curl fallback failed: exit={res.returncode} stderr={res.stderr[:200]}")
with _cb_lock: with _cb_lock:
_circuit_breaker[domain] = time.time() _circuit_breaker[domain] = time.time()
return _DummyResponse(500, "") return _DummyResponse(500, "")
except (subprocess.SubprocessError, ConnectionError, TimeoutError, OSError) as curl_e: except (subprocess.SubprocessError, ConnectionError, TimeoutError, OSError) as curl_e:
logger.error(f"bash curl fallback exception: {curl_e}") logger.error(f"curl fallback exception: {curl_e}")
with _cb_lock: with _cb_lock:
_circuit_breaker[domain] = time.time() _circuit_breaker[domain] = time.time()
return _DummyResponse(500, "") return _DummyResponse(500, "")
+10 -1
View File
@@ -15,6 +15,8 @@ _FEED_URL_REPLACEMENTS = {
"https://www.channelnewsasia.com/rssfeed/8395986": "https://www.channelnewsasia.com/api/v1/rss-outbound-feed?_format=xml", "https://www.channelnewsasia.com/rssfeed/8395986": "https://www.channelnewsasia.com/api/v1/rss-outbound-feed?_format=xml",
} }
_DEAD_FEED_URLS = { _DEAD_FEED_URLS = {
"https://www.reutersagency.com/feed/?best-topics=world",
"https://rsshub.app/apnews/topics/world-news",
"https://www3.nhk.or.jp/nhkworld/rss/world.xml", "https://www3.nhk.or.jp/nhkworld/rss/world.xml",
"https://focustaiwan.tw/rss", "https://focustaiwan.tw/rss",
"https://english.kyodonews.net/rss/news.xml", "https://english.kyodonews.net/rss/news.xml",
@@ -29,6 +31,11 @@ DEFAULT_FEEDS = [
{"name": "AlJazeera", "url": "https://www.aljazeera.com/xml/rss/all.xml", "weight": 2}, {"name": "AlJazeera", "url": "https://www.aljazeera.com/xml/rss/all.xml", "weight": 2},
{"name": "NYT", "url": "https://rss.nytimes.com/services/xml/rss/nyt/World.xml", "weight": 1}, {"name": "NYT", "url": "https://rss.nytimes.com/services/xml/rss/nyt/World.xml", "weight": 1},
{"name": "GDACS", "url": "https://www.gdacs.org/xml/rss.xml", "weight": 5}, {"name": "GDACS", "url": "https://www.gdacs.org/xml/rss.xml", "weight": 5},
{"name": "The War Zone", "url": "https://www.twz.com/feed", "weight": 4},
{"name": "Bellingcat", "url": "https://www.bellingcat.com/feed/", "weight": 4},
{"name": "Guardian", "url": "https://www.theguardian.com/world/rss", "weight": 3},
{"name": "TASS", "url": "https://tass.com/rss/v2.xml", "weight": 2},
{"name": "Xinhua", "url": "http://www.news.cn/english/rss/worldrss.xml", "weight": 2},
{"name": "CNA", "url": "https://www.channelnewsasia.com/api/v1/rss-outbound-feed?_format=xml", "weight": 3}, {"name": "CNA", "url": "https://www.channelnewsasia.com/api/v1/rss-outbound-feed?_format=xml", "weight": 3},
{"name": "Mercopress", "url": "https://en.mercopress.com/rss/", "weight": 3}, {"name": "Mercopress", "url": "https://en.mercopress.com/rss/", "weight": 3},
{"name": "SCMP", "url": "https://www.scmp.com/rss/91/feed", "weight": 4}, {"name": "SCMP", "url": "https://www.scmp.com/rss/91/feed", "weight": 4},
@@ -73,7 +80,9 @@ def get_feeds() -> list[dict]:
normalised = _normalise_feeds(feeds) normalised = _normalise_feeds(feeds)
if normalised != feeds: if normalised != feeds:
save_feeds(normalised) save_feeds(normalised)
return normalised if normalised:
return normalised
logger.warning("News feed configuration contained no usable feeds; falling back to defaults")
except (IOError, OSError, json.JSONDecodeError, ValueError) as e: except (IOError, OSError, json.JSONDecodeError, ValueError) as e:
logger.warning(f"Failed to read news feed config: {e}") logger.warning(f"Failed to read news feed config: {e}")
return list(DEFAULT_FEEDS) return list(DEFAULT_FEEDS)
+14 -3
View File
@@ -10,7 +10,8 @@ _cache: dict | None = None
_cache_ts: float = 0.0 _cache_ts: float = 0.0
_CACHE_TTL = 5.0 _CACHE_TTL = 5.0
_DEFAULTS = { _DEFAULTS = {
"enabled": False, "enabled": True,
"operator_disabled": False,
"timemachine_enabled": False, "timemachine_enabled": False,
} }
@@ -35,8 +36,16 @@ def read_node_settings() -> dict:
except Exception: except Exception:
result = {**_DEFAULTS, "updated_at": 0} result = {**_DEFAULTS, "updated_at": 0}
else: else:
operator_disabled = bool(data.get("operator_disabled", False))
raw_enabled = data.get("enabled", _DEFAULTS["enabled"])
# v0.9.7 initially wrote enabled:false as a default/offline state,
# which accidentally blocked InfoNet participation. Treat legacy
# false-without-marker as auto-enabled; only an explicit operator
# disable should keep the participant sync loop off.
enabled = False if operator_disabled else bool(raw_enabled or "operator_disabled" not in data)
result = { result = {
"enabled": bool(data.get("enabled", _DEFAULTS["enabled"])), "enabled": enabled,
"operator_disabled": operator_disabled,
"timemachine_enabled": bool(data.get("timemachine_enabled", _DEFAULTS["timemachine_enabled"])), "timemachine_enabled": bool(data.get("timemachine_enabled", _DEFAULTS["timemachine_enabled"])),
"updated_at": _safe_int(data.get("updated_at", 0) or 0), "updated_at": _safe_int(data.get("updated_at", 0) or 0),
} }
@@ -48,8 +57,10 @@ def read_node_settings() -> dict:
def write_node_settings(*, enabled: bool | None = None, timemachine_enabled: bool | None = None) -> dict: def write_node_settings(*, enabled: bool | None = None, timemachine_enabled: bool | None = None) -> dict:
DATA_DIR.mkdir(parents=True, exist_ok=True) DATA_DIR.mkdir(parents=True, exist_ok=True)
existing = read_node_settings() existing = read_node_settings()
next_enabled = bool(existing.get("enabled", _DEFAULTS["enabled"])) if enabled is None else bool(enabled)
payload = { payload = {
"enabled": bool(existing.get("enabled", _DEFAULTS["enabled"])) if enabled is None else bool(enabled), "enabled": next_enabled,
"operator_disabled": bool(existing.get("operator_disabled", _DEFAULTS["operator_disabled"])) if enabled is None else not next_enabled,
"timemachine_enabled": bool(existing.get("timemachine_enabled", _DEFAULTS["timemachine_enabled"])) if timemachine_enabled is None else bool(timemachine_enabled), "timemachine_enabled": bool(existing.get("timemachine_enabled", _DEFAULTS["timemachine_enabled"])) if timemachine_enabled is None else bool(timemachine_enabled),
"updated_at": int(time.time()), "updated_at": int(time.time()),
} }
+2
View File
@@ -104,6 +104,8 @@ def _match_prediction_markets(title: str, markets: list[dict]) -> dict | None:
"kalshi_pct": best_match.get("kalshi_pct"), "kalshi_pct": best_match.get("kalshi_pct"),
"consensus_pct": best_match.get("consensus_pct"), "consensus_pct": best_match.get("consensus_pct"),
"match_score": round(best_score, 2), "match_score": round(best_score, 2),
"slug": best_match.get("slug", ""),
"kalshi_ticker": best_match.get("kalshi_ticker", ""),
} }
+1 -1
View File
@@ -20,7 +20,7 @@ from cachetools import TTLCache
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
_SHODAN_BASE = "https://api.shodan.io" _SHODAN_BASE = "https://api.shodan.io"
_USER_AGENT = "ShadowBroker/0.9.7 local Shodan connector" _USER_AGENT = "ShadowBroker/0.9.75 local Shodan connector"
_REQUEST_TIMEOUT = 15 _REQUEST_TIMEOUT = 15
_MIN_INTERVAL_SECONDS = 1.05 # Shodan docs say API plans are rate limited to ~1 req/sec. _MIN_INTERVAL_SECONDS = 1.05 # Shodan docs say API plans are rate limited to ~1 req/sec.
_DEFAULT_SEARCH_PAGES = 1 _DEFAULT_SEARCH_PAGES = 1
+67 -26
View File
@@ -22,6 +22,12 @@ from collections import deque
from datetime import datetime, timezone from datetime import datetime, timezone
from services.config import get_settings from services.config import get_settings
from services.meshtastic_mqtt_settings import (
mqtt_bridge_enabled,
mqtt_connection_config,
mqtt_psk_hex,
mqtt_subscription_settings,
)
from services.mesh.meshtastic_topics import all_available_roots, build_subscription_topics, known_roots, parse_topic_metadata from services.mesh.meshtastic_topics import all_available_roots, build_subscription_topics, known_roots, parse_topic_metadata
logger = logging.getLogger("services.sigint") logger = logging.getLogger("services.sigint")
@@ -477,22 +483,13 @@ class MeshtasticBridge:
@staticmethod @staticmethod
def _mqtt_config() -> tuple[str, int, str, str]: def _mqtt_config() -> tuple[str, int, str, str]:
"""Return (broker, port, user, password) from settings.""" """Return (broker, port, user, password) from settings."""
try: return mqtt_connection_config()
s = get_settings()
return (
str(s.MESH_MQTT_BROKER or "mqtt.meshtastic.org"),
int(s.MESH_MQTT_PORT or 1883),
str(s.MESH_MQTT_USER or "meshdev"),
str(s.MESH_MQTT_PASS or "large4cats"),
)
except Exception:
return ("mqtt.meshtastic.org", 1883, "meshdev", "large4cats")
@classmethod @classmethod
def _resolve_psk(cls) -> bytes: def _resolve_psk(cls) -> bytes:
"""Return the PSK from config, or the default LongFast key if empty.""" """Return the PSK from config, or the default LongFast key if empty."""
try: try:
raw = str(getattr(get_settings(), "MESH_MQTT_PSK", "") or "").strip() raw = mqtt_psk_hex()
except Exception: except Exception:
raw = "" raw = ""
if not raw: if not raw:
@@ -506,6 +503,11 @@ class MeshtasticBridge:
self._thread: threading.Thread | None = None self._thread: threading.Thread | None = None
self._stop = threading.Event() self._stop = threading.Event()
self._client_id = self._build_client_id() self._client_id = self._build_client_id()
self._connected = False
self._last_error = ""
self._last_connected_at = 0.0
self._last_disconnected_at = 0.0
self._last_broker = ""
# Rate-limiter: sliding window of receive timestamps # Rate-limiter: sliding window of receive timestamps
self._rx_timestamps: deque[float] = deque() self._rx_timestamps: deque[float] = deque()
self._rx_dropped = 0 self._rx_dropped = 0
@@ -518,10 +520,11 @@ class MeshtasticBridge:
second client connects with the same id. Using a fixed id made separate second client connects with the same id. Using a fixed id made separate
ShadowBroker instances kick each other off the broker. ShadowBroker instances kick each other off the broker.
Includes the app version so the Meshtastic team can track our footprint. This is deliberately not tied to the user's public mesh address or
ShadowBroker node identity; it is only an MQTT session handle.
""" """
suffix = uuid.uuid4().hex[:8] suffix = uuid.uuid4().hex[:8]
return f"sb096-{suffix}" return f"meshchat-{suffix}"
def _dedupe_message( def _dedupe_message(
self, self,
@@ -544,7 +547,12 @@ class MeshtasticBridge:
def start(self): def start(self):
if self._thread and self._thread.is_alive(): if self._thread and self._thread.is_alive():
return if not self._stop.is_set():
return
self._thread.join(timeout=2.0)
if self._thread.is_alive():
logger.warning("Meshtastic MQTT bridge is still stopping; start deferred")
return
self._stop.clear() self._stop.clear()
self._thread = threading.Thread(target=self._run, daemon=True, name="mesh-bridge") self._thread = threading.Thread(target=self._run, daemon=True, name="mesh-bridge")
self._thread.start() self._thread.start()
@@ -552,13 +560,37 @@ class MeshtasticBridge:
def stop(self): def stop(self):
self._stop.set() self._stop.set()
self._connected = False
def is_running(self) -> bool:
return bool(self._thread and self._thread.is_alive() and not self._stop.is_set())
def status(self) -> dict:
broker, port, user, _pw = self._mqtt_config()
display_user = "" if user == "meshdev" else user
return {
"enabled": mqtt_bridge_enabled(),
"running": self.is_running(),
"connected": bool(self._connected),
"broker": broker,
"port": port,
"username": display_user,
"client_id": self._client_id,
"message_log_size": len(self.messages),
"signal_log_size": len(self.signals),
"last_error": self._last_error,
"last_broker": self._last_broker,
"last_connected_at": self._last_connected_at,
"last_disconnected_at": self._last_disconnected_at,
"rx_dropped": self._rx_dropped,
}
def _subscription_topics(self) -> list[str]: def _subscription_topics(self) -> list[str]:
settings = get_settings() extra_roots, extra_topics, include_defaults = mqtt_subscription_settings()
return build_subscription_topics( return build_subscription_topics(
extra_roots=str(getattr(settings, "MESH_MQTT_EXTRA_ROOTS", "") or ""), extra_roots=extra_roots,
extra_topics=str(getattr(settings, "MESH_MQTT_EXTRA_TOPICS", "") or ""), extra_topics=extra_topics,
include_defaults=bool(getattr(settings, "MESH_MQTT_INCLUDE_DEFAULT_ROOTS", True)), include_defaults=include_defaults,
) )
def _run(self): def _run(self):
@@ -582,6 +614,9 @@ class MeshtasticBridge:
def _on_connect(client, userdata, flags, rc): def _on_connect(client, userdata, flags, rc):
if rc == 0: if rc == 0:
self._connected = True
self._last_error = ""
self._last_connected_at = time.time()
logger.info( logger.info(
"Meshtastic MQTT connected (%s), subscribing to %s", "Meshtastic MQTT connected (%s), subscribing to %s",
self._client_id, self._client_id,
@@ -590,6 +625,8 @@ class MeshtasticBridge:
for topic in topics: for topic in topics:
client.subscribe(topic, qos=0) client.subscribe(topic, qos=0)
else: else:
self._connected = False
self._last_error = f"connect_refused:{rc}"
logger.error( logger.error(
"Meshtastic MQTT connection refused (%s): rc=%s", "Meshtastic MQTT connection refused (%s): rc=%s",
self._client_id, self._client_id,
@@ -597,7 +634,10 @@ class MeshtasticBridge:
) )
def _on_disconnect(client, userdata, rc): def _on_disconnect(client, userdata, rc):
self._connected = False
self._last_disconnected_at = time.time()
if rc != 0: if rc != 0:
self._last_error = f"disconnect:{rc}"
logger.warning( logger.warning(
"Meshtastic MQTT disconnected unexpectedly (%s, rc=%s), will auto-reconnect", "Meshtastic MQTT disconnected unexpectedly (%s, rc=%s), will auto-reconnect",
self._client_id, self._client_id,
@@ -607,6 +647,7 @@ class MeshtasticBridge:
logger.info("Meshtastic MQTT disconnected cleanly (%s)", self._client_id) logger.info("Meshtastic MQTT disconnected cleanly (%s)", self._client_id)
broker, port, user, pw = self._mqtt_config() broker, port, user, pw = self._mqtt_config()
self._last_broker = f"{broker}:{port}"
client = mqtt.Client(client_id=self._client_id, protocol=mqtt.MQTTv311) client = mqtt.Client(client_id=self._client_id, protocol=mqtt.MQTTv311)
client.username_pw_set(user, pw) client.username_pw_set(user, pw)
client.on_connect = _on_connect client.on_connect = _on_connect
@@ -645,9 +686,6 @@ class MeshtasticBridge:
def _on_message(self, client, userdata, msg): def _on_message(self, client, userdata, msg):
"""Parse Meshtastic MQTT messages — protobuf + AES decryption.""" """Parse Meshtastic MQTT messages — protobuf + AES decryption."""
try: try:
if self._rate_limited():
return
payload = msg.payload payload = msg.payload
topic = msg.topic topic = msg.topic
@@ -655,6 +693,8 @@ class MeshtasticBridge:
if "/json/" in topic: if "/json/" in topic:
try: try:
data = json.loads(payload) data = json.loads(payload)
if self._rate_limited():
return
self._ingest_data(data, topic) self._ingest_data(data, topic)
return return
except (json.JSONDecodeError, UnicodeDecodeError): except (json.JSONDecodeError, UnicodeDecodeError):
@@ -687,6 +727,8 @@ class MeshtasticBridge:
} }
) )
else: else:
if self._rate_limited():
return
self._ingest_data(data, topic) self._ingest_data(data, topic)
except Exception as e: except Exception as e:
@@ -1011,7 +1053,7 @@ class SIGINTGrid:
self._started = True self._started = True
self.aprs.start() self.aprs.start()
try: try:
mqtt_enabled = bool(getattr(get_settings(), "MESH_MQTT_ENABLED", False)) mqtt_enabled = mqtt_bridge_enabled()
except Exception: except Exception:
mqtt_enabled = False mqtt_enabled = False
if mqtt_enabled: if mqtt_enabled:
@@ -1123,13 +1165,12 @@ class SIGINTGrid:
ch = msg.get("channel", "LongFast") ch = msg.get("channel", "LongFast")
channel_msgs[ch] = channel_msgs.get(ch, 0) + 1 channel_msgs[ch] = channel_msgs.get(ch, 0) + 1
extra_roots, _extra_topics, include_defaults = mqtt_subscription_settings()
return { return {
"regions": regions, "regions": regions,
"roots": roots, "roots": roots,
"known_roots": known_roots( "known_roots": known_roots(extra_roots, include_defaults=include_defaults),
str(getattr(get_settings(), "MESH_MQTT_EXTRA_ROOTS", "") or ""),
include_defaults=bool(getattr(get_settings(), "MESH_MQTT_INCLUDE_DEFAULT_ROOTS", True)),
),
"all_roots": all_available_roots(), "all_roots": all_available_roots(),
"channel_messages": channel_msgs, "channel_messages": channel_msgs,
"total_nodes": len(seen_callsigns), "total_nodes": len(seen_callsigns),
+69 -94
View File
@@ -1,13 +1,9 @@
"""Tor Hidden Service auto-provisioner. """Tor hidden-service auto-provisioner.
Manages a Tor hidden service that points to the local ShadowBroker backend. Manages a Tor hidden service that points to the local ShadowBroker backend.
Tor is started as a subprocess with a generated torrc no manual config needed. Tor is started as a subprocess with a generated torrc. Windows source installs
Auto-installs the Tor Expert Bundle on Windows if not present. can download the Tor Expert Bundle into backend/data without admin rights.
Docker images should already include the `tor` package.
Usage:
from services.tor_hidden_service import tor_service
status = tor_service.start() # -> {"ok": True, "onion_address": "http://xxxx.onion:8000"}
tor_service.stop()
""" """
from __future__ import annotations from __future__ import annotations
@@ -31,31 +27,33 @@ HOSTNAME_PATH = TOR_DIR / "hidden_service" / "hostname"
TOR_DATA_DIR = TOR_DIR / "data" TOR_DATA_DIR = TOR_DIR / "data"
PIDFILE_PATH = TOR_DIR / "tor.pid" PIDFILE_PATH = TOR_DIR / "tor.pid"
# Bundled Tor install location (inside our data dir so no admin rights needed) # Bundled Tor install location (inside data dir so no admin rights are needed).
TOR_INSTALL_DIR = TOR_DIR / "tor_bin" TOR_INSTALL_DIR = TOR_DIR / "tor_bin"
# How long to wait for Tor to generate the hostname file
_STARTUP_TIMEOUT_S = 90 _STARTUP_TIMEOUT_S = 90
_POLL_INTERVAL_S = 1.0 _POLL_INTERVAL_S = 1.0
# Tor Expert Bundle download URL (Windows x86_64) # Windows x86_64 Tor Expert Bundle URLs. Keep a fallback so first-run
_TOR_EXPERT_BUNDLE_URL = "https://dist.torproject.org/torbrowser/15.0.8/tor-expert-bundle-windows-x86_64-15.0.8.tar.gz" # onboarding does not break when Tor rotates point releases.
_TOR_EXPERT_BUNDLE_URLS = [
"https://dist.torproject.org/torbrowser/15.0.11/tor-expert-bundle-windows-x86_64-15.0.11.tar.gz",
"https://dist.torproject.org/torbrowser/15.0.8/tor-expert-bundle-windows-x86_64-15.0.8.tar.gz",
]
def _find_tor_binary() -> str | None: def _find_tor_binary() -> str | None:
"""Locate the tor binary on the system, including our bundled install.""" """Locate the tor binary on the system, including our bundled install."""
# Check our bundled install first
bundled = TOR_INSTALL_DIR / "tor" / "tor.exe" bundled = TOR_INSTALL_DIR / "tor" / "tor.exe"
if bundled.exists(): if bundled.exists():
return str(bundled) return str(bundled)
# Also check for extracted layout variants
for sub in TOR_INSTALL_DIR.rglob("tor.exe"): for sub in TOR_INSTALL_DIR.rglob("tor.exe"):
return str(sub) return str(sub)
tor = shutil.which("tor") tor = shutil.which("tor")
if tor: if tor:
return tor return tor
# Common locations on Windows
for candidate in [ for candidate in [
r"C:\Program Files\Tor Browser\Browser\TorBrowser\Tor\tor.exe", r"C:\Program Files\Tor Browser\Browser\TorBrowser\Tor\tor.exe",
r"C:\Program Files (x86)\Tor Browser\Browser\TorBrowser\Tor\tor.exe", r"C:\Program Files (x86)\Tor Browser\Browser\TorBrowser\Tor\tor.exe",
@@ -67,77 +65,65 @@ def _find_tor_binary() -> str | None:
def _auto_install_tor() -> str | None: def _auto_install_tor() -> str | None:
"""Download and extract the Tor Expert Bundle. Returns path to tor binary or None.""" """Install or download Tor when it is safe to do so."""
if os.name != "nt": if os.name != "nt":
# On Linux/Mac, try package manager # In Docker this should already be baked into the image. For source
try: # installs we avoid unattended sudo prompts from a web request path.
if shutil.which("apt-get"): logger.warning("Tor is not installed. Install the tor package or use the Docker image with Tor baked in.")
subprocess.run(["sudo", "apt-get", "install", "-y", "tor"], check=True, capture_output=True, timeout=120) return None
elif shutil.which("brew"):
subprocess.run(["brew", "install", "tor"], check=True, capture_output=True, timeout=120)
elif shutil.which("pacman"):
subprocess.run(["sudo", "pacman", "-S", "--noconfirm", "tor"], check=True, capture_output=True, timeout=120)
else:
logger.warning("No supported package manager found for auto-install")
return None
return shutil.which("tor")
except Exception as exc:
logger.error("Failed to auto-install Tor via package manager: %s", exc)
return None
# Windows: download Tor Expert Bundle (no admin needed)
TOR_INSTALL_DIR.mkdir(parents=True, exist_ok=True) TOR_INSTALL_DIR.mkdir(parents=True, exist_ok=True)
archive_path = TOR_INSTALL_DIR / "tor-expert-bundle.tar.gz" for bundle_url in _TOR_EXPERT_BUNDLE_URLS:
archive_path = TOR_INSTALL_DIR / "tor-expert-bundle.tar.gz"
try:
logger.info("Downloading Tor Expert Bundle over HTTPS from dist.torproject.org...")
urlretrieve(_TOR_EXPERT_BUNDLE_URL, str(archive_path))
# Verify SHA-256 of the downloaded archive
sha256_url = _TOR_EXPERT_BUNDLE_URL + ".sha256sum"
sha256_file = TOR_INSTALL_DIR / "sha256sum.txt"
try: try:
urlretrieve(sha256_url, str(sha256_file)) logger.info("Downloading Tor Expert Bundle over HTTPS from %s...", bundle_url)
expected_hash = sha256_file.read_text().strip().split()[0].lower() urlretrieve(bundle_url, str(archive_path))
import hashlib
actual_hash = hashlib.sha256(archive_path.read_bytes()).hexdigest().lower()
sha256_file.unlink(missing_ok=True)
if actual_hash != expected_hash:
logger.error("SHA-256 MISMATCH — download may be compromised! Expected %s, got %s", expected_hash, actual_hash)
archive_path.unlink(missing_ok=True)
return None
logger.info("SHA-256 verified: %s", actual_hash[:16] + "...")
except Exception as hash_err:
# If we can't fetch the hash file, warn but proceed (HTTPS provides baseline integrity)
logger.warning("Could not verify SHA-256 (hash file unavailable): %s — proceeding with HTTPS-only verification", hash_err)
logger.info("Download complete, extracting...") sha256_url = bundle_url + ".sha256sum"
sha256_file = TOR_INSTALL_DIR / "sha256sum.txt"
try:
urlretrieve(sha256_url, str(sha256_file))
expected_hash = sha256_file.read_text().strip().split()[0].lower()
import hashlib
# Extract .tar.gz with path traversal protection actual_hash = hashlib.sha256(archive_path.read_bytes()).hexdigest().lower()
import tarfile sha256_file.unlink(missing_ok=True)
with tarfile.open(str(archive_path), "r:gz") as tar: if actual_hash != expected_hash:
for member in tar.getmembers(): logger.error("SHA-256 mismatch for Tor download. Expected %s, got %s", expected_hash, actual_hash)
member_path = (TOR_INSTALL_DIR / member.name).resolve()
if not str(member_path).startswith(str(TOR_INSTALL_DIR.resolve())):
logger.error("Tar path traversal blocked: %s", member.name)
archive_path.unlink(missing_ok=True) archive_path.unlink(missing_ok=True)
return None continue
tar.extractall(path=str(TOR_INSTALL_DIR)) logger.info("SHA-256 verified: %s", actual_hash[:16] + "...")
except Exception as hash_err:
logger.warning(
"Could not verify SHA-256 (hash file unavailable): %s; proceeding with HTTPS-only verification",
hash_err,
)
# Clean up archive logger.info("Download complete, extracting...")
archive_path.unlink(missing_ok=True) import tarfile
# Find the tor.exe in extracted files with tarfile.open(str(archive_path), "r:gz") as tar:
for p in TOR_INSTALL_DIR.rglob("tor.exe"): for member in tar.getmembers():
logger.info("Tor installed at: %s", p) member_path = (TOR_INSTALL_DIR / member.name).resolve()
return str(p) if not str(member_path).startswith(str(TOR_INSTALL_DIR.resolve())):
logger.error("Tar path traversal blocked: %s", member.name)
archive_path.unlink(missing_ok=True)
return None
tar.extractall(path=str(TOR_INSTALL_DIR))
logger.error("tor.exe not found after extraction") archive_path.unlink(missing_ok=True)
return None
except Exception as exc: for p in TOR_INSTALL_DIR.rglob("tor.exe"):
logger.error("Failed to download/extract Tor: %s", exc) logger.info("Tor installed at: %s", p)
archive_path.unlink(missing_ok=True) return str(p)
return None
logger.error("tor.exe not found after extracting %s", bundle_url)
except Exception as exc:
logger.error("Failed to download/extract Tor from %s: %s", bundle_url, exc)
finally:
archive_path.unlink(missing_ok=True)
return None
class TorHiddenService: class TorHiddenService:
@@ -150,7 +136,6 @@ class TorHiddenService:
self._running = False self._running = False
self._error: str = "" self._error: str = ""
# Check if we already have a hostname from a previous run
if HOSTNAME_PATH.exists(): if HOSTNAME_PATH.exists():
try: try:
hostname = HOSTNAME_PATH.read_text().strip() hostname = HOSTNAME_PATH.read_text().strip()
@@ -198,19 +183,20 @@ class TorHiddenService:
self._error = "" self._error = ""
tor_bin = _find_tor_binary() tor_bin = _find_tor_binary()
if not tor_bin: if not tor_bin:
logger.info("Tor not found, attempting auto-install...") logger.info("Tor not found, attempting bootstrap...")
tor_bin = _auto_install_tor() tor_bin = _auto_install_tor()
if not tor_bin: if not tor_bin:
self._error = "Failed to auto-install Tor. Please install it manually." self._error = (
"Could not prepare Tor automatically. Check network access to dist.torproject.org "
"or install Tor, then try again."
)
return {"ok": False, "detail": self._error} return {"ok": False, "detail": self._error}
# Create directories
TOR_DIR.mkdir(parents=True, exist_ok=True) TOR_DIR.mkdir(parents=True, exist_ok=True)
TOR_DATA_DIR.mkdir(parents=True, exist_ok=True) TOR_DATA_DIR.mkdir(parents=True, exist_ok=True)
hidden_service_dir = TOR_DIR / "hidden_service" hidden_service_dir = TOR_DIR / "hidden_service"
hidden_service_dir.mkdir(parents=True, exist_ok=True) hidden_service_dir.mkdir(parents=True, exist_ok=True)
# On non-Windows, Tor requires strict permissions on HiddenServiceDir
if os.name != "nt": if os.name != "nt":
try: try:
os.chmod(str(hidden_service_dir), 0o700) os.chmod(str(hidden_service_dir), 0o700)
@@ -218,19 +204,15 @@ class TorHiddenService:
except OSError: except OSError:
pass pass
# Write torrc — enables both hidden service (inbound) and SOCKS proxy
# (outbound) so the mesh/wormhole system can route node-to-node
# traffic through Tor as well.
torrc_content = ( torrc_content = (
f"DataDirectory {TOR_DATA_DIR.as_posix()}\n" f"DataDirectory {TOR_DATA_DIR.as_posix()}\n"
f"HiddenServiceDir {hidden_service_dir.as_posix()}\n" f"HiddenServiceDir {hidden_service_dir.as_posix()}\n"
f"HiddenServicePort {target_port} 127.0.0.1:{target_port}\n" f"HiddenServicePort {target_port} 127.0.0.1:{target_port}\n"
f"SocksPort 9050\n" "SocksPort 9050\n"
f"Log notice stderr\n" "Log notice stderr\n"
) )
TORRC_PATH.write_text(torrc_content, encoding="utf-8") TORRC_PATH.write_text(torrc_content, encoding="utf-8")
# Start Tor
try: try:
self._process = subprocess.Popen( self._process = subprocess.Popen(
[tor_bin, "-f", str(TORRC_PATH)], [tor_bin, "-f", str(TORRC_PATH)],
@@ -245,15 +227,12 @@ class TorHiddenService:
logger.error(self._error) logger.error(self._error)
return {"ok": False, "detail": self._error} return {"ok": False, "detail": self._error}
# Wait for hostname file to appear
deadline = time.monotonic() + _STARTUP_TIMEOUT_S deadline = time.monotonic() + _STARTUP_TIMEOUT_S
while time.monotonic() < deadline: while time.monotonic() < deadline:
if self._process.poll() is not None: if self._process.poll() is not None:
# Tor exited prematurely
stdout = self._process.stdout.read() if self._process.stdout else "" stdout = self._process.stdout.read() if self._process.stdout else ""
self._error = f"Tor exited with code {self._process.returncode}" self._error = f"Tor exited with code {self._process.returncode}"
if stdout: if stdout:
# Get last few lines for error context
lines = stdout.strip().split("\n") lines = stdout.strip().split("\n")
self._error += ": " + " | ".join(lines[-3:]) self._error += ": " + " | ".join(lines[-3:])
self._running = False self._running = False
@@ -273,7 +252,6 @@ class TorHiddenService:
time.sleep(_POLL_INTERVAL_S) time.sleep(_POLL_INTERVAL_S)
# Timeout
self._error = f"Tor did not generate hostname within {_STARTUP_TIMEOUT_S}s" self._error = f"Tor did not generate hostname within {_STARTUP_TIMEOUT_S}s"
self.stop() self.stop()
return {"ok": False, "detail": self._error} return {"ok": False, "detail": self._error}
@@ -292,10 +270,7 @@ class TorHiddenService:
pass pass
self._process = None self._process = None
self._running = False self._running = False
# Keep the onion_address — it persists across restarts
# since the key is stored in hidden_service_dir
return {"ok": True, "detail": "stopped"} return {"ok": True, "detail": "stopped"}
# Singleton
tor_service = TorHiddenService() tor_service = TorHiddenService()
+1 -1
View File
@@ -24,7 +24,7 @@ from cachetools import TTLCache
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
_FINNHUB_BASE = "https://finnhub.io/api/v1" _FINNHUB_BASE = "https://finnhub.io/api/v1"
_USER_AGENT = "ShadowBroker/0.9.7 Finnhub connector" _USER_AGENT = "ShadowBroker/0.9.75 Finnhub connector"
_REQUEST_TIMEOUT = 12 _REQUEST_TIMEOUT = 12
_MIN_INTERVAL_SECONDS = 0.35 # Stay well under 60 calls/min _MIN_INTERVAL_SECONDS = 0.35 # Stay well under 60 calls/min
+11 -1
View File
@@ -225,6 +225,11 @@ def _installed() -> bool:
def _pid_alive(pid: int) -> bool: def _pid_alive(pid: int) -> bool:
if pid <= 0: if pid <= 0:
return False return False
if os.name == "nt":
# Windows PIDs are reused and os.kill(pid, 0) is not a reliable
# ownership check. A persisted wormhole_status.json PID from an older
# run must never be treated as a process we own.
return False
try: try:
os.kill(pid, 0) os.kill(pid, 0)
except OSError: except OSError:
@@ -268,7 +273,12 @@ def _current_runtime_state() -> dict[str, Any]:
pid = int(_PROCESS.pid or 0) pid = int(_PROCESS.pid or 0)
elif _pid_alive(pid): elif _pid_alive(pid):
running = True running = True
elif _probe_ready(timeout_s=0.35):
running = True
pid = 0
ready = running and _probe_ready() ready = running and _probe_ready()
if not running:
pid = 0
transport_active = status.get("transport_active", "") if ready else "" transport_active = status.get("transport_active", "") if ready else ""
proxy_active = status.get("proxy_active", "") if ready else "" proxy_active = status.get("proxy_active", "") if ready else ""
effective_transport = str(transport_active or settings.get("transport", "direct") or "direct").lower() effective_transport = str(transport_active or settings.get("transport", "direct") or "direct").lower()
@@ -489,7 +499,7 @@ def disconnect_wormhole(*, reason: str = "disconnect") -> dict[str, Any]:
_PROCESS.kill() _PROCESS.kill()
except Exception: except Exception:
pass pass
elif _pid_alive(pid): elif os.name != "nt" and _pid_alive(pid):
try: try:
os.kill(pid, signal.SIGTERM) os.kill(pid, signal.SIGTERM)
except Exception: except Exception:
@@ -52,7 +52,9 @@ def test_refresh_node_peer_store_promotes_manifest_peers_to_sync_only(tmp_path,
monkeypatch.setenv("MESH_BOOTSTRAP_SIGNER_PUBLIC_KEY", manifest_pub) monkeypatch.setenv("MESH_BOOTSTRAP_SIGNER_PUBLIC_KEY", manifest_pub)
monkeypatch.setenv("MESH_BOOTSTRAP_MANIFEST_PATH", str(manifest_path)) monkeypatch.setenv("MESH_BOOTSTRAP_MANIFEST_PATH", str(manifest_path))
monkeypatch.setenv("MESH_RELAY_PEERS", "https://operator.example") monkeypatch.setenv("MESH_RELAY_PEERS", "https://operator.example")
monkeypatch.setenv("MESH_BOOTSTRAP_SEED_PEERS", "")
monkeypatch.setenv("MESH_DEFAULT_SYNC_PEERS", "") monkeypatch.setenv("MESH_DEFAULT_SYNC_PEERS", "")
monkeypatch.setenv("MESH_INFONET_ALLOW_CLEARNET_SYNC", "true")
get_settings.cache_clear() get_settings.cache_clear()
try: try:
@@ -74,7 +76,7 @@ def test_refresh_node_peer_store_promotes_manifest_peers_to_sync_only(tmp_path,
assert [record.peer_url for record in store.records_for_bucket("push")] == ["https://operator.example"] assert [record.peer_url for record in store.records_for_bucket("push")] == ["https://operator.example"]
def test_refresh_node_peer_store_adds_default_seed_as_pull_only_peer(tmp_path, monkeypatch): def test_refresh_node_peer_store_adds_bootstrap_seed_as_pull_only_peer(tmp_path, monkeypatch):
import main import main
from services.config import get_settings from services.config import get_settings
from services.mesh import mesh_peer_store as peer_store_mod from services.mesh import mesh_peer_store as peer_store_mod
@@ -82,7 +84,9 @@ def test_refresh_node_peer_store_adds_default_seed_as_pull_only_peer(tmp_path, m
peer_store_path = tmp_path / "peer_store.json" peer_store_path = tmp_path / "peer_store.json"
monkeypatch.setattr(peer_store_mod, "DEFAULT_PEER_STORE_PATH", peer_store_path) monkeypatch.setattr(peer_store_mod, "DEFAULT_PEER_STORE_PATH", peer_store_path)
monkeypatch.setenv("MESH_RELAY_PEERS", "") monkeypatch.setenv("MESH_RELAY_PEERS", "")
monkeypatch.setenv("MESH_DEFAULT_SYNC_PEERS", "https://node.shadowbroker.info") monkeypatch.setenv("MESH_BOOTSTRAP_SEED_PEERS", "https://node.shadowbroker.info")
monkeypatch.setenv("MESH_DEFAULT_SYNC_PEERS", "")
monkeypatch.setenv("MESH_INFONET_ALLOW_CLEARNET_SYNC", "true")
monkeypatch.setenv("MESH_BOOTSTRAP_SIGNER_PUBLIC_KEY", "") monkeypatch.setenv("MESH_BOOTSTRAP_SIGNER_PUBLIC_KEY", "")
get_settings.cache_clear() get_settings.cache_clear()
@@ -94,6 +98,7 @@ def test_refresh_node_peer_store_adds_default_seed_as_pull_only_peer(tmp_path, m
get_settings.cache_clear() get_settings.cache_clear()
assert snapshot["manifest_loaded"] is False assert snapshot["manifest_loaded"] is False
assert snapshot["bootstrap_seed_peer_count"] == 1
assert snapshot["default_sync_peer_count"] == 1 assert snapshot["default_sync_peer_count"] == 1
assert snapshot["bootstrap_peer_count"] == 1 assert snapshot["bootstrap_peer_count"] == 1
assert snapshot["sync_peer_count"] == 1 assert snapshot["sync_peer_count"] == 1
@@ -107,6 +112,36 @@ def test_refresh_node_peer_store_adds_default_seed_as_pull_only_peer(tmp_path, m
assert store.records_for_bucket("sync")[0].source == "bundle" assert store.records_for_bucket("sync")[0].source == "bundle"
def test_refresh_node_peer_store_suppresses_clearnet_seed_by_default(tmp_path, monkeypatch):
import main
from services.config import get_settings
from services.mesh import mesh_peer_store as peer_store_mod
peer_store_path = tmp_path / "peer_store.json"
monkeypatch.setattr(peer_store_mod, "DEFAULT_PEER_STORE_PATH", peer_store_path)
monkeypatch.setenv("MESH_RELAY_PEERS", "")
monkeypatch.setenv("MESH_BOOTSTRAP_SEED_PEERS", "https://node.shadowbroker.info")
monkeypatch.setenv("MESH_DEFAULT_SYNC_PEERS", "")
monkeypatch.delenv("MESH_INFONET_ALLOW_CLEARNET_SYNC", raising=False)
monkeypatch.setenv("MESH_BOOTSTRAP_SIGNER_PUBLIC_KEY", "")
get_settings.cache_clear()
try:
snapshot = main._refresh_node_peer_store(now=1_750_000_000)
store = peer_store_mod.PeerStore(peer_store_path)
store.load()
finally:
get_settings.cache_clear()
assert snapshot["private_transport_required"] is True
assert snapshot["skipped_clearnet_peer_count"] == 1
assert snapshot["bootstrap_peer_count"] == 0
assert snapshot["sync_peer_count"] == 0
assert "no clearnet sync fallback" in snapshot["last_bootstrap_error"]
assert store.records_for_bucket("bootstrap") == []
assert store.records_for_bucket("sync") == []
def test_verify_peer_push_hmac_requires_allowlisted_peer(monkeypatch): def test_verify_peer_push_hmac_requires_allowlisted_peer(monkeypatch):
import hashlib import hashlib
import hmac import hmac
@@ -172,13 +207,19 @@ def test_infonet_status_includes_node_runtime_snapshot(monkeypatch):
def test_public_sync_cycle_allows_first_node_without_peers(tmp_path, monkeypatch): def test_public_sync_cycle_allows_first_node_without_peers(tmp_path, monkeypatch):
import main import main
from services.config import get_settings
from services.mesh import mesh_peer_store as peer_store_mod from services.mesh import mesh_peer_store as peer_store_mod
peer_store_path = tmp_path / "peer_store.json" peer_store_path = tmp_path / "peer_store.json"
monkeypatch.setattr(peer_store_mod, "DEFAULT_PEER_STORE_PATH", peer_store_path) monkeypatch.setattr(peer_store_mod, "DEFAULT_PEER_STORE_PATH", peer_store_path)
monkeypatch.setattr(main, "_participant_node_enabled", lambda: True) monkeypatch.setattr(main, "_participant_node_enabled", lambda: True)
monkeypatch.setenv("MESH_INFONET_ALLOW_CLEARNET_SYNC", "true")
get_settings.cache_clear()
result = main._run_public_sync_cycle() try:
result = main._run_public_sync_cycle()
finally:
get_settings.cache_clear()
assert result.last_outcome == "solo" assert result.last_outcome == "solo"
assert result.last_error == "" assert result.last_error == ""
@@ -0,0 +1,54 @@
import importlib
def test_meshtastic_mqtt_settings_redacts_secrets(tmp_path, monkeypatch):
monkeypatch.setenv("SB_DATA_DIR", str(tmp_path))
from services import meshtastic_mqtt_settings
settings = importlib.reload(meshtastic_mqtt_settings)
saved = settings.write_meshtastic_mqtt_settings(
enabled=True,
broker="mqtt.example.test",
port=1884,
username="mesh-user",
password="mesh-pass",
psk="001122",
include_default_roots=False,
extra_roots="EU,US",
)
redacted = settings.redacted_meshtastic_mqtt_settings(saved)
assert saved["password"] == "mesh-pass"
assert saved["psk"] == "001122"
assert redacted["enabled"] is True
assert redacted["broker"] == "mqtt.example.test"
assert redacted["port"] == 1884
assert redacted["username"] == "mesh-user"
assert redacted["has_password"] is True
assert redacted["has_psk"] is True
assert "password" not in redacted
assert "psk" not in redacted
assert settings.mqtt_connection_config() == ("mqtt.example.test", 1884, "mesh-user", "mesh-pass")
assert settings.mqtt_bridge_enabled() is True
assert settings.mqtt_psk_hex() == "001122"
assert settings.mqtt_subscription_settings() == ("EU,US", "", False)
def test_meshtastic_mqtt_settings_hide_public_defaults(tmp_path, monkeypatch):
monkeypatch.setenv("SB_DATA_DIR", str(tmp_path))
from services import meshtastic_mqtt_settings
settings = importlib.reload(meshtastic_mqtt_settings)
saved = settings.write_meshtastic_mqtt_settings(
enabled=True,
broker="mqtt.meshtastic.org",
username="",
password="",
)
redacted = settings.redacted_meshtastic_mqtt_settings(saved)
assert redacted["username"] == ""
assert redacted["uses_default_credentials"] is True
assert settings.mqtt_connection_config() == ("mqtt.meshtastic.org", 1883, "meshdev", "large4cats")
@@ -0,0 +1,21 @@
from services.mesh.meshtastic_topics import build_subscription_topics, known_roots, parse_topic_metadata
def test_default_subscription_is_longfast_only():
assert build_subscription_topics() == ["msh/US/2/e/LongFast/#"]
assert known_roots() == ["US"]
def test_extra_roots_are_longfast_only():
assert build_subscription_topics(extra_roots="EU_868,ANZ") == [
"msh/US/2/e/LongFast/#",
"msh/EU_868/2/e/LongFast/#",
"msh/ANZ/2/e/LongFast/#",
]
def test_parse_longfast_topic_root():
meta = parse_topic_metadata("msh/US/2/e/LongFast/!12345678")
assert meta["region"] == "US"
assert meta["root"] == "US"
assert meta["channel"] == "LongFast"
+36 -1
View File
@@ -7,9 +7,44 @@ def test_node_settings_roundtrip(tmp_path, monkeypatch):
monkeypatch.setattr(node_settings, "_cache_ts", 0.0) monkeypatch.setattr(node_settings, "_cache_ts", 0.0)
initial = node_settings.read_node_settings() initial = node_settings.read_node_settings()
disabled = node_settings.write_node_settings(enabled=False)
updated = node_settings.write_node_settings(enabled=True) updated = node_settings.write_node_settings(enabled=True)
reread = node_settings.read_node_settings() reread = node_settings.read_node_settings()
assert initial["enabled"] is False assert initial["enabled"] is True
assert initial["operator_disabled"] is False
assert disabled["enabled"] is False
assert disabled["operator_disabled"] is True
assert updated["enabled"] is True assert updated["enabled"] is True
assert updated["operator_disabled"] is False
assert reread["enabled"] is True assert reread["enabled"] is True
def test_legacy_disabled_node_settings_auto_enable(tmp_path, monkeypatch):
from services import node_settings
settings_path = tmp_path / "node.json"
settings_path.write_text('{"enabled": false, "updated_at": 123}', encoding="utf-8")
monkeypatch.setattr(node_settings, "NODE_FILE", settings_path)
monkeypatch.setattr(node_settings, "_cache", None)
monkeypatch.setattr(node_settings, "_cache_ts", 0.0)
reread = node_settings.read_node_settings()
assert reread["enabled"] is True
assert reread["operator_disabled"] is False
def test_explicit_operator_disabled_stays_disabled(tmp_path, monkeypatch):
from services import node_settings
settings_path = tmp_path / "node.json"
settings_path.write_text('{"enabled": false, "operator_disabled": true, "updated_at": 123}', encoding="utf-8")
monkeypatch.setattr(node_settings, "NODE_FILE", settings_path)
monkeypatch.setattr(node_settings, "_cache", None)
monkeypatch.setattr(node_settings, "_cache_ts", 0.0)
reread = node_settings.read_node_settings()
assert reread["enabled"] is False
assert reread["operator_disabled"] is True
@@ -2,7 +2,7 @@
Tests prove: Tests prove:
- Docker no longer auto-allows raw fallback - Docker no longer auto-allows raw fallback
- Non-Windows with no secure secret and no raw opt-in fails closed - Non-Windows with no secure secret generates a local passphrase file
- Non-Windows with MESH_SECURE_STORAGE_SECRET works (passphrase provider) - Non-Windows with MESH_SECURE_STORAGE_SECRET works (passphrase provider)
- Passphrase-protected envelopes round-trip correctly (master + domain) - Passphrase-protected envelopes round-trip correctly (master + domain)
- Raw-to-passphrase migration works when secret is supplied - Raw-to-passphrase migration works when secret is supplied
@@ -64,10 +64,10 @@ class TestDockerNoAutoRawFallback:
assert mesh_secure_storage._raw_fallback_allowed() is True assert mesh_secure_storage._raw_fallback_allowed() is True
class TestFailClosedWithoutSecret: class TestGeneratedLocalSecretWithoutOperatorSecret:
"""Non-Windows with no secret and no raw opt-in must fail closed.""" """Non-Windows with no supplied secret generates a local passphrase file."""
def test_master_key_creation_fails_closed(self, tmp_path, monkeypatch): def test_master_key_creation_uses_generated_local_secret(self, tmp_path, monkeypatch):
from services.mesh import mesh_secure_storage from services.mesh import mesh_secure_storage
from services import config as config_mod from services import config as config_mod
@@ -76,6 +76,7 @@ class TestFailClosedWithoutSecret:
monkeypatch.setattr(mesh_secure_storage, "_is_windows", lambda: False) monkeypatch.setattr(mesh_secure_storage, "_is_windows", lambda: False)
monkeypatch.delenv("PYTEST_CURRENT_TEST", raising=False) monkeypatch.delenv("PYTEST_CURRENT_TEST", raising=False)
monkeypatch.delenv("MESH_SECURE_STORAGE_SECRET", raising=False) monkeypatch.delenv("MESH_SECURE_STORAGE_SECRET", raising=False)
monkeypatch.delenv("MESH_SECURE_STORAGE_SECRET_FILE", raising=False)
monkeypatch.setattr( monkeypatch.setattr(
config_mod, config_mod,
"get_settings", "get_settings",
@@ -86,10 +87,14 @@ class TestFailClosedWithoutSecret:
) )
_reset(mesh_secure_storage) _reset(mesh_secure_storage)
with pytest.raises(mesh_secure_storage.SecureStorageError, match="MESH_SECURE_STORAGE_SECRET"): key = mesh_secure_storage._load_master_key()
mesh_secure_storage._load_master_key() assert len(key) == 32
assert (tmp_path / "secure_storage_secret.key").exists()
envelope = json.loads((tmp_path / "master.key").read_text(encoding="utf-8"))
assert envelope["provider"] == "passphrase"
assert "key" not in envelope
def test_domain_key_creation_fails_closed(self, tmp_path, monkeypatch): def test_domain_key_creation_uses_generated_local_secret(self, tmp_path, monkeypatch):
from services.mesh import mesh_secure_storage from services.mesh import mesh_secure_storage
from services import config as config_mod from services import config as config_mod
@@ -98,6 +103,7 @@ class TestFailClosedWithoutSecret:
monkeypatch.setattr(mesh_secure_storage, "_is_windows", lambda: False) monkeypatch.setattr(mesh_secure_storage, "_is_windows", lambda: False)
monkeypatch.delenv("PYTEST_CURRENT_TEST", raising=False) monkeypatch.delenv("PYTEST_CURRENT_TEST", raising=False)
monkeypatch.delenv("MESH_SECURE_STORAGE_SECRET", raising=False) monkeypatch.delenv("MESH_SECURE_STORAGE_SECRET", raising=False)
monkeypatch.delenv("MESH_SECURE_STORAGE_SECRET_FILE", raising=False)
monkeypatch.setattr( monkeypatch.setattr(
config_mod, config_mod,
"get_settings", "get_settings",
@@ -108,8 +114,12 @@ class TestFailClosedWithoutSecret:
) )
_reset(mesh_secure_storage) _reset(mesh_secure_storage)
with pytest.raises(mesh_secure_storage.SecureStorageError, match="MESH_SECURE_STORAGE_SECRET"): key = mesh_secure_storage._load_domain_key("test_domain", base_dir=tmp_path)
mesh_secure_storage._load_domain_key("test_domain", base_dir=tmp_path) assert len(key) == 32
assert (tmp_path / "secure_storage_secret.key").exists()
envelope = json.loads((tmp_path / "_domain_keys" / "test_domain.key").read_text(encoding="utf-8"))
assert envelope["provider"] == "passphrase"
assert "key" not in envelope
class TestPassphraseProvider: class TestPassphraseProvider:
@@ -311,7 +321,7 @@ class TestWrongPassphraseFails:
), ),
) )
with pytest.raises(mesh_secure_storage.SecureStorageError, match="MESH_SECURE_STORAGE_SECRET is not set"): with pytest.raises(mesh_secure_storage.SecureStorageError, match="Failed to unwrap"):
mesh_secure_storage._load_master_key() mesh_secure_storage._load_master_key()
@@ -517,6 +527,7 @@ class TestGetStorageSecret:
from services import config as config_mod from services import config as config_mod
monkeypatch.delenv("MESH_SECURE_STORAGE_SECRET", raising=False) monkeypatch.delenv("MESH_SECURE_STORAGE_SECRET", raising=False)
monkeypatch.setattr(mesh_secure_storage, "_is_windows", lambda: True)
monkeypatch.setattr( monkeypatch.setattr(
config_mod, config_mod,
"get_settings", "get_settings",
@@ -524,6 +535,27 @@ class TestGetStorageSecret:
) )
assert mesh_secure_storage._get_storage_secret() is None assert mesh_secure_storage._get_storage_secret() is None
def test_generates_local_secret_file_on_non_windows(self, tmp_path, monkeypatch):
from services.mesh import mesh_secure_storage
from services import config as config_mod
secret_file = tmp_path / "generated_secret.key"
monkeypatch.delenv("MESH_SECURE_STORAGE_SECRET", raising=False)
monkeypatch.delenv("PYTEST_CURRENT_TEST", raising=False)
monkeypatch.setenv("MESH_SECURE_STORAGE_SECRET_FILE", str(secret_file))
monkeypatch.setattr(mesh_secure_storage, "_is_windows", lambda: False)
monkeypatch.setattr(
config_mod,
"get_settings",
lambda: SimpleNamespace(MESH_SECURE_STORAGE_SECRET=""),
)
first = mesh_secure_storage._get_storage_secret()
second = mesh_secure_storage._get_storage_secret()
assert first
assert second == first
assert secret_file.read_text(encoding="utf-8").strip() == first
def test_falls_back_to_config(self, monkeypatch): def test_falls_back_to_config(self, monkeypatch):
from services.mesh import mesh_secure_storage from services.mesh import mesh_secure_storage
from services import config as config_mod from services import config as config_mod
+38
View File
@@ -0,0 +1,38 @@
import os
def test_save_api_keys_persists_write_only(tmp_path, monkeypatch):
from services import api_settings
key_store = tmp_path / "operator_api_keys.env"
backend_env = tmp_path / ".env"
monkeypatch.setattr(api_settings, "OPERATOR_KEYS_ENV_PATH", key_store)
monkeypatch.setattr(api_settings, "ENV_PATH", backend_env)
monkeypatch.delenv("OPENSKY_CLIENT_ID", raising=False)
result = api_settings.save_api_keys(
{
"OPENSKY_CLIENT_ID": "client-id-value",
"NOT_ALLOWED": "ignore-me",
}
)
assert result["ok"] is True
assert result["updated"] == ["OPENSKY_CLIENT_ID"]
assert "client-id-value" not in str(result)
assert os.environ["OPENSKY_CLIENT_ID"] == "client-id-value"
assert 'OPENSKY_CLIENT_ID="client-id-value"' in key_store.read_text(encoding="utf-8")
assert "NOT_ALLOWED" not in key_store.read_text(encoding="utf-8")
def test_persisted_api_keys_load_when_process_env_blank(tmp_path, monkeypatch):
from services import api_settings
key_store = tmp_path / "operator_api_keys.env"
key_store.write_text('AIS_API_KEY="saved-ais-key"\n', encoding="utf-8")
monkeypatch.setattr(api_settings, "OPERATOR_KEYS_ENV_PATH", key_store)
monkeypatch.setenv("AIS_API_KEY", "")
api_settings.load_persisted_api_keys_into_environ()
assert os.environ["AIS_API_KEY"] == "saved-ais-key"
+6 -2
View File
@@ -145,6 +145,10 @@ class TestFeedConfig:
def test_new_east_asia_feeds_present(self): def test_new_east_asia_feeds_present(self):
names = {f["name"] for f in DEFAULT_FEEDS} names = {f["name"] for f in DEFAULT_FEEDS}
expected = {"FocusTaiwan", "Kyodo", "SCMP", "The Diplomat", "Stars and Stripes", expected = {"SCMP", "The Diplomat", "Yonhap", "Asia Times", "Defense News", "Japan Times"}
"Yonhap", "Nikkei Asia", "Taipei Times", "Asia Times", "Defense News", "Japan Times"}
assert expected.issubset(names) assert expected.issubset(names)
def test_known_dead_feeds_are_not_defaulted(self):
urls = {f["url"] for f in DEFAULT_FEEDS}
assert "https://www.reutersagency.com/feed/?best-topics=world" not in urls
assert "https://rsshub.app/apnews/topics/world-news" not in urls
+38 -16
View File
@@ -86,6 +86,18 @@ class TestRequireLocalOperator:
def test_rfc1918_172_blocked_without_key(self): def test_rfc1918_172_blocked_without_key(self):
assert self._call_with_host("172.16.0.5") == 403 assert self._call_with_host("172.16.0.5") == 403
def test_docker_bridge_blocked_without_compose_opt_in(self):
with patch.dict("os.environ", {"SHADOWBROKER_TRUST_DOCKER_BRIDGE_LOCAL_OPERATOR": ""}):
assert self._call_with_host("172.18.0.3") == 403
def test_docker_bridge_passes_with_compose_opt_in(self):
with patch.dict("os.environ", {"SHADOWBROKER_TRUST_DOCKER_BRIDGE_LOCAL_OPERATOR": "1"}):
assert self._call_with_host("172.18.0.3") == 200
def test_lan_ip_still_blocked_with_compose_opt_in(self):
with patch.dict("os.environ", {"SHADOWBROKER_TRUST_DOCKER_BRIDGE_LOCAL_OPERATOR": "1"}):
assert self._call_with_host("192.168.1.100") == 403
def test_rfc1918_192168_blocked_without_key(self): def test_rfc1918_192168_blocked_without_key(self):
assert self._call_with_host("192.168.1.100") == 403 assert self._call_with_host("192.168.1.100") == 403
@@ -100,7 +112,14 @@ class TestRequireLocalOperator:
# _validate_peer_push_secret — startup enforcement # _validate_peer_push_secret — startup enforcement
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
_KNOWN_COMPROMISED = "Mv63UvLfwqOEVWeRBXjA8MtFl2nEkkhUlLYVHiX1Zzo" _KNOWN_COMPROMISED = "".join(
[
"Mv63UvLfwq",
"OEVWeRBXjA",
"8MtFl2nEkk",
"hUlLYVHiX1Zzo",
]
)
class TestValidatePeerPushSecret: class TestValidatePeerPushSecret:
@@ -114,16 +133,17 @@ class TestValidatePeerPushSecret:
with patch("main.get_settings", return_value=mock_settings): with patch("main.get_settings", return_value=mock_settings):
return _validate_peer_push_secret return _validate_peer_push_secret
def test_known_default_causes_exit(self): def test_known_default_auto_generates_replacement(self):
from auth import _validate_peer_push_secret from auth import _validate_peer_push_secret
mock_settings = MagicMock() mock_settings = MagicMock()
mock_settings.MESH_PEER_PUSH_SECRET = _KNOWN_COMPROMISED mock_settings.MESH_PEER_PUSH_SECRET = _KNOWN_COMPROMISED
with patch("auth.get_settings", return_value=mock_settings): with (
with pytest.raises(SystemExit) as exc_info: patch("auth.get_settings", return_value=mock_settings),
_validate_peer_push_secret() patch("auth._auto_generate_peer_push_secret", return_value="replacement-secret-value"),
assert exc_info.value.code == 1 ):
_validate_peer_push_secret()
def test_empty_secret_does_not_exit_without_peers(self): def test_empty_secret_does_not_exit_without_peers(self):
from auth import _validate_peer_push_secret from auth import _validate_peer_push_secret
@@ -137,7 +157,7 @@ class TestValidatePeerPushSecret:
with patch("auth.get_settings", return_value=mock_settings): with patch("auth.get_settings", return_value=mock_settings):
_validate_peer_push_secret() # no exception = pass _validate_peer_push_secret() # no exception = pass
def test_empty_secret_with_peers_causes_exit(self): def test_empty_secret_with_peers_auto_generates_replacement(self):
from auth import _validate_peer_push_secret from auth import _validate_peer_push_secret
mock_settings = MagicMock() mock_settings = MagicMock()
@@ -146,12 +166,13 @@ class TestValidatePeerPushSecret:
mock_settings.MESH_RNS_PEERS = "" mock_settings.MESH_RNS_PEERS = ""
mock_settings.MESH_RNS_ENABLED = False mock_settings.MESH_RNS_ENABLED = False
with patch("auth.get_settings", return_value=mock_settings): with (
with pytest.raises(SystemExit) as exc_info: patch("auth.get_settings", return_value=mock_settings),
_validate_peer_push_secret() patch("auth._auto_generate_peer_push_secret", return_value="replacement-secret-value"),
assert exc_info.value.code == 1 ):
_validate_peer_push_secret()
def test_short_secret_with_peers_causes_exit(self): def test_short_secret_with_peers_auto_generates_replacement(self):
from auth import _validate_peer_push_secret from auth import _validate_peer_push_secret
mock_settings = MagicMock() mock_settings = MagicMock()
@@ -160,10 +181,11 @@ class TestValidatePeerPushSecret:
mock_settings.MESH_RNS_PEERS = "" mock_settings.MESH_RNS_PEERS = ""
mock_settings.MESH_RNS_ENABLED = False mock_settings.MESH_RNS_ENABLED = False
with patch("auth.get_settings", return_value=mock_settings): with (
with pytest.raises(SystemExit) as exc_info: patch("auth.get_settings", return_value=mock_settings),
_validate_peer_push_secret() patch("auth._auto_generate_peer_push_secret", return_value="replacement-secret-value"),
assert exc_info.value.code == 1 ):
_validate_peer_push_secret()
def test_valid_secret_passes(self): def test_valid_secret_passes(self):
from auth import _validate_peer_push_secret from auth import _validate_peer_push_secret
+2 -2
View File
@@ -1,12 +1,12 @@
{ {
"name": "@shadowbroker/desktop-shell", "name": "@shadowbroker/desktop-shell",
"version": "0.9.7", "version": "0.9.75",
"lockfileVersion": 3, "lockfileVersion": 3,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "@shadowbroker/desktop-shell", "name": "@shadowbroker/desktop-shell",
"version": "0.9.7", "version": "0.9.75",
"devDependencies": { "devDependencies": {
"typescript": "^5.6.0" "typescript": "^5.6.0"
} }
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@shadowbroker/desktop-shell", "name": "@shadowbroker/desktop-shell",
"version": "0.9.7", "version": "0.9.75",
"private": true, "private": true,
"description": "ShadowBroker desktop shell packaging, runtime bridge, and release tooling", "description": "ShadowBroker desktop shell packaging, runtime bridge, and release tooling",
"scripts": { "scripts": {
+25
View File
@@ -9,6 +9,7 @@ $repoRoot = Resolve-Path (Join-Path $scriptDir "..\..")
$frontendDir = Join-Path $repoRoot "frontend" $frontendDir = Join-Path $repoRoot "frontend"
$frontendOut = Join-Path $frontendDir "out" $frontendOut = Join-Path $frontendDir "out"
$srcTauriDir = Join-Path $scriptDir "src-tauri" $srcTauriDir = Join-Path $scriptDir "src-tauri"
$tauriConfigPath = Join-Path $srcTauriDir "tauri.conf.json"
$companionDir = Join-Path $srcTauriDir "companion-www" $companionDir = Join-Path $srcTauriDir "companion-www"
$backendRuntimeDir = Join-Path $srcTauriDir "backend-runtime" $backendRuntimeDir = Join-Path $srcTauriDir "backend-runtime"
$iconsScript = Join-Path $scriptDir "scripts\generate-icons.cjs" $iconsScript = Join-Path $scriptDir "scripts\generate-icons.cjs"
@@ -43,6 +44,18 @@ function Invoke-External {
} }
} }
function Write-Utf8NoBom {
param(
[Parameter(Mandatory = $true)]
[string]$Path,
[Parameter(Mandatory = $true)]
[string]$Content
)
$encoding = New-Object System.Text.UTF8Encoding($false)
[System.IO.File]::WriteAllText($Path, $Content, $encoding)
}
foreach ($tool in @("cargo", "npm", "node")) { foreach ($tool in @("cargo", "npm", "node")) {
if (-not (Get-Command $tool -ErrorAction SilentlyContinue)) { if (-not (Get-Command $tool -ErrorAction SilentlyContinue)) {
throw "$tool is required for desktop packaging." throw "$tool is required for desktop packaging."
@@ -107,6 +120,7 @@ Write-Host " -> $fileCount files"
Write-Host "" Write-Host ""
Push-Location $srcTauriDir Push-Location $srcTauriDir
$tauriConfigBackup = $null
try { try {
if (-not $env:SHADOWBROKER_BACKEND_URL) { if (-not $env:SHADOWBROKER_BACKEND_URL) {
$env:SHADOWBROKER_BACKEND_URL = "http://127.0.0.1:8000" $env:SHADOWBROKER_BACKEND_URL = "http://127.0.0.1:8000"
@@ -131,6 +145,14 @@ try {
Write-Host "Updater signing: enabled" Write-Host "Updater signing: enabled"
} else { } else {
Write-Host "Updater signing: disabled (set TAURI_SIGNING_PRIVATE_KEY_PATH to emit update signatures)" Write-Host "Updater signing: disabled (set TAURI_SIGNING_PRIVATE_KEY_PATH to emit update signatures)"
$tauriConfigBackup = Get-Content -LiteralPath $tauriConfigPath -Raw
$tauriConfig = $tauriConfigBackup | ConvertFrom-Json
if ($tauriConfig.bundle.createUpdaterArtifacts) {
$tauriConfig.bundle.createUpdaterArtifacts = $false
$tauriConfig |
ConvertTo-Json -Depth 100 |
ForEach-Object { Write-Utf8NoBom -Path $tauriConfigPath -Content ($_ + "`n") }
}
} }
Write-Host "" Write-Host ""
@@ -147,5 +169,8 @@ try {
} }
} }
finally { finally {
if ($null -ne $tauriConfigBackup) {
Write-Utf8NoBom -Path $tauriConfigPath -Content $tauriConfigBackup
}
Pop-Location Pop-Location
} }
@@ -2,11 +2,13 @@
const fs = require('node:fs'); const fs = require('node:fs');
const path = require('node:path'); const path = require('node:path');
const { spawnSync } = require('node:child_process');
const scriptDir = __dirname; const scriptDir = __dirname;
const tauriDir = path.resolve(scriptDir, '..'); const tauriDir = path.resolve(scriptDir, '..');
const repoRoot = path.resolve(tauriDir, '..', '..'); const repoRoot = path.resolve(tauriDir, '..', '..');
const backendDir = path.join(repoRoot, 'backend'); const backendDir = path.join(repoRoot, 'backend');
const privacyCoreDir = path.join(repoRoot, 'privacy-core');
const outputDir = path.join(tauriDir, 'src-tauri', 'backend-runtime'); const outputDir = path.join(tauriDir, 'src-tauri', 'backend-runtime');
const venvMarkerPath = path.join(backendDir, '.venv-dir'); const venvMarkerPath = path.join(backendDir, '.venv-dir');
const releaseAttestationPath = path.join(backendDir, 'data', 'release_attestation.json'); const releaseAttestationPath = path.join(backendDir, 'data', 'release_attestation.json');
@@ -19,14 +21,21 @@ const stagedReleaseAttestationPath = path.join(
const excludedNames = new Set([ const excludedNames = new Set([
'.env', '.env',
'.pytest_cache', '.pytest_cache',
'.ruff_cache',
'__pycache__', '__pycache__',
'backend.egg-info', 'backend.egg-info',
'build', 'build',
'data', 'data',
'tests', 'tests',
'timemachine',
]); ]);
const excludedFiles = new Set([ const excludedFiles = new Set([
'.env.example',
'ais_cache.json',
'carrier_cache.json',
'cctv.db',
'dm_token_pepper.key',
'pytest.ini', 'pytest.ini',
]); ]);
@@ -77,15 +86,58 @@ function ensureRuntimePrereqs() {
} }
} }
function privacyCoreArtifactName() {
if (process.platform === 'win32') return 'privacy_core.dll';
if (process.platform === 'darwin') return 'libprivacy_core.dylib';
return 'libprivacy_core.so';
}
function privacyCoreArtifactPath() {
return path.join(privacyCoreDir, 'target', 'release', privacyCoreArtifactName());
}
function ensurePrivacyCoreArtifact() {
const artifact = privacyCoreArtifactPath();
if (fs.existsSync(artifact)) {
return artifact;
}
console.log('privacy-core release library missing; building it for desktop packaging...');
const result = spawnSync(
'cargo',
['build', '--release', '--manifest-path', path.join(privacyCoreDir, 'Cargo.toml')],
{
cwd: repoRoot,
env: process.env,
stdio: 'inherit',
},
);
if (result.error || result.status !== 0) {
throw new Error(
'Failed to build privacy-core release library. Install Rust/Cargo and rerun the desktop build.',
);
}
if (!fs.existsSync(artifact)) {
throw new Error(`privacy-core build completed but artifact is missing: ${artifact}`);
}
return artifact;
}
function stageBackendRuntime() { function stageBackendRuntime() {
fs.rmSync(outputDir, { recursive: true, force: true }); fs.rmSync(outputDir, { recursive: true, force: true });
fs.cpSync(backendDir, outputDir, { fs.cpSync(backendDir, outputDir, {
recursive: true, recursive: true,
filter: shouldCopy, filter: shouldCopy,
}); });
stagePrivacyCoreArtifact();
stageReleaseAttestation(); stageReleaseAttestation();
} }
function stagePrivacyCoreArtifact() {
const artifact = ensurePrivacyCoreArtifact();
const stagedPath = path.join(outputDir, path.basename(artifact));
fs.copyFileSync(artifact, stagedPath);
}
function stageReleaseAttestation() { function stageReleaseAttestation() {
if (!fs.existsSync(releaseAttestationPath)) { if (!fs.existsSync(releaseAttestationPath)) {
console.warn(`backend-runtime staged without release attestation: ${releaseAttestationPath}`); console.warn(`backend-runtime staged without release attestation: ${releaseAttestationPath}`);
@@ -43,6 +43,18 @@ function prepareBuildTree() {
filter: shouldCopy, filter: shouldCopy,
}); });
const stagedLayoutPath = path.join(buildFrontendDir, 'src', 'app', 'layout.tsx');
if (fs.existsSync(stagedLayoutPath)) {
const layoutSource = fs.readFileSync(stagedLayoutPath, 'utf8');
fs.writeFileSync(
stagedLayoutPath,
layoutSource
.replace(/\n\/\/ The dashboard is a live local runtime[\s\S]*?client polling ever hydrates\.\n/g, '\n')
.replace(/\nexport const dynamic = ['"]force-dynamic['"];\n/g, '\n')
.replace(/\nexport const revalidate = 0;\n/g, '\n'),
);
}
const liveNodeModules = path.join(frontendDir, 'node_modules'); const liveNodeModules = path.join(frontendDir, 'node_modules');
const stagedNodeModules = path.join(buildFrontendDir, 'node_modules'); const stagedNodeModules = path.join(buildFrontendDir, 'node_modules');
if (!fs.existsSync(liveNodeModules)) { if (!fs.existsSync(liveNodeModules)) {
+1 -1
View File
@@ -4201,7 +4201,7 @@ dependencies = [
[[package]] [[package]]
name = "shadowbroker-tauri-shell" name = "shadowbroker-tauri-shell"
version = "0.9.7" version = "0.9.75"
dependencies = [ dependencies = [
"axum", "axum",
"base64 0.22.1", "base64 0.22.1",
@@ -1,6 +1,6 @@
[package] [package]
name = "shadowbroker-tauri-shell" name = "shadowbroker-tauri-shell"
version = "0.9.7" version = "0.9.75"
edition = "2021" edition = "2021"
[build-dependencies] [build-dependencies]
@@ -91,7 +91,8 @@ pub async fn ensure_and_start_managed_backend(
.open(&stderr_log) .open(&stderr_log)
.map_err(|e| format!("managed_backend_stderr_log_failed:{e}"))?; .map_err(|e| format!("managed_backend_stderr_log_failed:{e}"))?;
let mut child = Command::new(&python_bin) let mut command = Command::new(&python_bin);
command
.current_dir(&runtime_root) .current_dir(&runtime_root)
.arg("-m") .arg("-m")
.arg("uvicorn") .arg("uvicorn")
@@ -103,7 +104,13 @@ pub async fn ensure_and_start_managed_backend(
.arg("--timeout-keep-alive") .arg("--timeout-keep-alive")
.arg("120") .arg("120")
.env("PYTHONUNBUFFERED", "1") .env("PYTHONUNBUFFERED", "1")
.env("SB_DATA_DIR", data_dir.as_os_str()) .env("SB_DATA_DIR", data_dir.as_os_str());
if let Some(privacy_core_lib) = bundled_privacy_core_lib(&runtime_root) {
command.env("PRIVACY_CORE_LIB", privacy_core_lib.as_os_str());
}
let mut child = command
.stdout(Stdio::from(stdout)) .stdout(Stdio::from(stdout))
.stderr(Stdio::from(stderr)) .stderr(Stdio::from(stderr))
.spawn() .spawn()
@@ -191,6 +198,18 @@ fn sync_release_attestation(bundled_root: &Path, install_root: &Path) -> Result<
Ok(()) Ok(())
} }
fn bundled_privacy_core_lib(runtime_root: &Path) -> Option<PathBuf> {
let file_name = if cfg!(target_os = "windows") {
"privacy_core.dll"
} else if cfg!(target_os = "macos") {
"libprivacy_core.dylib"
} else {
"libprivacy_core.so"
};
let candidate = runtime_root.join(file_name);
candidate.exists().then_some(candidate)
}
fn release_attestation_path(root: &Path) -> PathBuf { fn release_attestation_path(root: &Path) -> PathBuf {
RELEASE_ATTESTATION_RELATIVE_PATH RELEASE_ATTESTATION_RELATIVE_PATH
.iter() .iter()
@@ -1,7 +1,7 @@
{ {
"$schema": "https://schema.tauri.app/config/2", "$schema": "https://schema.tauri.app/config/2",
"productName": "ShadowBroker", "productName": "ShadowBroker",
"version": "0.9.7", "version": "0.9.75",
"identifier": "com.shadowbroker.desktop", "identifier": "com.shadowbroker.desktop",
"build": { "build": {
"frontendDist": "../../../frontend/out", "frontendDist": "../../../frontend/out",
@@ -38,7 +38,7 @@
}, },
"plugins": { "plugins": {
"updater": { "updater": {
"pubkey": "dW50cnVzdGVkIGNvbW1lbnQ6IG1pbmlzaWduIHB1YmxpYyBrZXk6IDJDMUU1NkRENjNCNTI5RjUKUldUMUtiVmozVlllTEd0STJlMGtORUxUWHlGQ2V0ZXM3Z1BOc3hwc0pUK1c3dlplcWc2OFpKd3oK", "pubkey": "dW50cnVzdGVkIGNvbW1lbnQ6IG1pbmlzaWduIHB1YmxpYyBrZXk6IEUxODExMjQ4MkJBMThFNTgKUldSWWpxRXJTQktCNFF3ZXNQbndUK0pVWUEwNDNuajcrUGI3ZEI4TWtDUDlQdHhudmlHUkNjQUUK",
"endpoints": [ "endpoints": [
"https://github.com/BigBodyCobain/Shadowbroker/releases/latest/download/latest.json" "https://github.com/BigBodyCobain/Shadowbroker/releases/latest/download/latest.json"
], ],
-1
View File
@@ -21,7 +21,6 @@ services:
resources: resources:
limits: limits:
memory: 2G memory: 2G
cpus: '2'
volumes: volumes:
relay_data: relay_data:
+26 -10
View File
@@ -11,22 +11,38 @@ services:
image: ghcr.io/bigbodycobain/shadowbroker-backend:latest image: ghcr.io/bigbodycobain/shadowbroker-backend:latest
container_name: shadowbroker-backend container_name: shadowbroker-backend
ports: ports:
- "${BIND:-127.0.0.1}:8000:8000" - "${BIND:-127.0.0.1}:${BACKEND_PORT:-8000}:8000"
environment: environment:
- AIS_API_KEY=${AIS_API_KEY} - AIS_API_KEY=${AIS_API_KEY:-}
- OPENSKY_CLIENT_ID=${OPENSKY_CLIENT_ID} - OPENSKY_CLIENT_ID=${OPENSKY_CLIENT_ID:-}
- OPENSKY_CLIENT_SECRET=${OPENSKY_CLIENT_SECRET} - OPENSKY_CLIENT_SECRET=${OPENSKY_CLIENT_SECRET:-}
- LTA_ACCOUNT_KEY=${LTA_ACCOUNT_KEY} - LTA_ACCOUNT_KEY=${LTA_ACCOUNT_KEY:-}
- ADMIN_KEY=${ADMIN_KEY:-} - ADMIN_KEY=${ADMIN_KEY:-}
- FINNHUB_API_KEY=${FINNHUB_API_KEY:-} - FINNHUB_API_KEY=${FINNHUB_API_KEY:-}
# Override allowed CORS origins (comma-separated). Auto-detects LAN IPs if empty. # Override allowed CORS origins (comma-separated). Auto-detects LAN IPs if empty.
- CORS_ORIGINS=${CORS_ORIGINS:-} - CORS_ORIGINS=${CORS_ORIGINS:-}
# Default public Infonet seed used for pull-only sync by fresh installs. # Private Infonet bootstrap seeds. Seeds are discovery hints, not fixed roots.
- MESH_DEFAULT_SYNC_PEERS=${MESH_DEFAULT_SYNC_PEERS:-https://node.shadowbroker.info} - MESH_BOOTSTRAP_SEED_PEERS=${MESH_BOOTSTRAP_SEED_PEERS:-http://gqpbunqbgtkcqilvclm3xrkt3zowjyl3s62kkktvojgvxzizamvbrqid.onion:8000}
- MESH_DEFAULT_SYNC_PEERS=${MESH_DEFAULT_SYNC_PEERS:-}
# Operator-trusted sync/push peers. Leave empty unless you control the peer secret on both sides. # Operator-trusted sync/push peers. Leave empty unless you control the peer secret on both sides.
- MESH_RELAY_PEERS=${MESH_RELAY_PEERS:-} - MESH_RELAY_PEERS=${MESH_RELAY_PEERS:-}
# Shared transport auth for operator peer push. Must be set to a unique secret per deployment. # Shared transport auth for operator peer push. Must be set to a unique secret per deployment.
- MESH_PEER_PUSH_SECRET=${MESH_PEER_PUSH_SECRET} - MESH_PEER_PUSH_SECRET=${MESH_PEER_PUSH_SECRET:-}
# Meshtastic MQTT is opt-in to avoid passive load on the public broker.
# Set MESH_MQTT_ENABLED=true in .env only when this node should join live MQTT.
- MESH_MQTT_ENABLED=${MESH_MQTT_ENABLED:-false}
- MESH_MQTT_BROKER=${MESH_MQTT_BROKER:-mqtt.meshtastic.org}
- MESH_MQTT_PORT=${MESH_MQTT_PORT:-1883}
- MESH_MQTT_USER=${MESH_MQTT_USER:-meshdev}
- MESH_MQTT_PASS=${MESH_MQTT_PASS:-large4cats}
- MESH_MQTT_PSK=${MESH_MQTT_PSK:-}
- MESH_MQTT_INCLUDE_DEFAULT_ROOTS=${MESH_MQTT_INCLUDE_DEFAULT_ROOTS:-true}
- MESH_MQTT_EXTRA_ROOTS=${MESH_MQTT_EXTRA_ROOTS:-}
- MESH_MQTT_EXTRA_TOPICS=${MESH_MQTT_EXTRA_TOPICS:-}
- MESHTASTIC_OPERATOR_CALLSIGN=${MESHTASTIC_OPERATOR_CALLSIGN:-}
# The bundled Docker UI talks to the backend across Docker's private bridge.
# Treat that bridge as local operator access while ports remain bound to 127.0.0.1 by default.
- SHADOWBROKER_TRUST_DOCKER_BRIDGE_LOCAL_OPERATOR=${SHADOWBROKER_TRUST_DOCKER_BRIDGE_LOCAL_OPERATOR:-1}
volumes: volumes:
- backend_data:/app/data - backend_data:/app/data
restart: unless-stopped restart: unless-stopped
@@ -39,7 +55,7 @@ services:
deploy: deploy:
resources: resources:
limits: limits:
memory: 2G memory: ${BACKEND_MEMORY_LIMIT:-4G}
cpus: '2' cpus: '2'
frontend: frontend:
@@ -56,7 +72,7 @@ services:
condition: service_healthy condition: service_healthy
restart: unless-stopped restart: unless-stopped
healthcheck: healthcheck:
test: ["CMD", "wget", "-q", "--spider", "http://localhost:3000/"] test: ["CMD", "wget", "-q", "--spider", "http://127.0.0.1:3000/"]
interval: 30s interval: 30s
timeout: 10s timeout: 10s
retries: 3 retries: 3
+136 -397
View File
@@ -1,12 +1,12 @@
{ {
"name": "frontend", "name": "frontend",
"version": "0.9.7", "version": "0.9.75",
"lockfileVersion": 3, "lockfileVersion": 3,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "frontend", "name": "frontend",
"version": "0.9.7", "version": "0.9.75",
"dependencies": { "dependencies": {
"@mapbox/point-geometry": "^1.1.0", "@mapbox/point-geometry": "^1.1.0",
"@tauri-apps/plugin-process": "^2.3.1", "@tauri-apps/plugin-process": "^2.3.1",
@@ -33,9 +33,9 @@
"@vitest/coverage-v8": "^4.1.0", "@vitest/coverage-v8": "^4.1.0",
"concurrently": "^9.2.1", "concurrently": "^9.2.1",
"eslint": "^9", "eslint": "^9",
"eslint-config-next": "16.1.6", "eslint-config-next": "16.2.4",
"jsdom": "^28.1.0", "jsdom": "^28.1.0",
"prettier": "^3.3.3", "prettier": "^3.8.3",
"tailwindcss": "^4", "tailwindcss": "^4",
"typescript": "^5", "typescript": "^5",
"vitest": "^4.1.0" "vitest": "^4.1.0"
@@ -1362,9 +1362,9 @@
"license": "MIT" "license": "MIT"
}, },
"node_modules/@next/eslint-plugin-next": { "node_modules/@next/eslint-plugin-next": {
"version": "16.1.6", "version": "16.2.4",
"resolved": "https://registry.npmjs.org/@next/eslint-plugin-next/-/eslint-plugin-next-16.1.6.tgz", "resolved": "https://registry.npmjs.org/@next/eslint-plugin-next/-/eslint-plugin-next-16.2.4.tgz",
"integrity": "sha512-/Qq3PTagA6+nYVfryAtQ7/9FEr/6YVyvOtl6rZnGsbReGLf0jZU6gkpr1FuChAQpvV46a78p4cmHOVP8mbfSMQ==", "integrity": "sha512-tOX826JJ96gYK/go18sPUgMq9FK1tqxBFfUCEufJb5XIkWFFmpgU7mahJANKGkHs7F41ir3tReJ3Lv5La0RvhA==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
@@ -1870,49 +1870,49 @@
} }
}, },
"node_modules/@tailwindcss/node": { "node_modules/@tailwindcss/node": {
"version": "4.2.1", "version": "4.2.4",
"resolved": "https://registry.npmjs.org/@tailwindcss/node/-/node-4.2.1.tgz", "resolved": "https://registry.npmjs.org/@tailwindcss/node/-/node-4.2.4.tgz",
"integrity": "sha512-jlx6sLk4EOwO6hHe1oCGm1Q4AN/s0rSrTTPBGPM0/RQ6Uylwq17FuU8IeJJKEjtc6K6O07zsvP+gDO6MMWo7pg==", "integrity": "sha512-Ai7+yQPxz3ddrDQzFfBKdHEVBg0w3Zl83jnjuwxnZOsnH9pGn93QHQtpU0p/8rYWxvbFZHneni6p1BSLK4DkGA==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@jridgewell/remapping": "^2.3.5", "@jridgewell/remapping": "^2.3.5",
"enhanced-resolve": "^5.19.0", "enhanced-resolve": "^5.19.0",
"jiti": "^2.6.1", "jiti": "^2.6.1",
"lightningcss": "1.31.1", "lightningcss": "1.32.0",
"magic-string": "^0.30.21", "magic-string": "^0.30.21",
"source-map-js": "^1.2.1", "source-map-js": "^1.2.1",
"tailwindcss": "4.2.1" "tailwindcss": "4.2.4"
} }
}, },
"node_modules/@tailwindcss/oxide": { "node_modules/@tailwindcss/oxide": {
"version": "4.2.1", "version": "4.2.4",
"resolved": "https://registry.npmjs.org/@tailwindcss/oxide/-/oxide-4.2.1.tgz", "resolved": "https://registry.npmjs.org/@tailwindcss/oxide/-/oxide-4.2.4.tgz",
"integrity": "sha512-yv9jeEFWnjKCI6/T3Oq50yQEOqmpmpfzG1hcZsAOaXFQPfzWprWrlHSdGPEF3WQTi8zu8ohC9Mh9J470nT5pUw==", "integrity": "sha512-9El/iI069DKDSXwTvB9J4BwdO5JhRrOweGaK25taBAvBXyXqJAX+Jqdvs8r8gKpsI/1m0LeJLyQYTf/WLrBT1Q==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"engines": { "engines": {
"node": ">= 20" "node": ">= 20"
}, },
"optionalDependencies": { "optionalDependencies": {
"@tailwindcss/oxide-android-arm64": "4.2.1", "@tailwindcss/oxide-android-arm64": "4.2.4",
"@tailwindcss/oxide-darwin-arm64": "4.2.1", "@tailwindcss/oxide-darwin-arm64": "4.2.4",
"@tailwindcss/oxide-darwin-x64": "4.2.1", "@tailwindcss/oxide-darwin-x64": "4.2.4",
"@tailwindcss/oxide-freebsd-x64": "4.2.1", "@tailwindcss/oxide-freebsd-x64": "4.2.4",
"@tailwindcss/oxide-linux-arm-gnueabihf": "4.2.1", "@tailwindcss/oxide-linux-arm-gnueabihf": "4.2.4",
"@tailwindcss/oxide-linux-arm64-gnu": "4.2.1", "@tailwindcss/oxide-linux-arm64-gnu": "4.2.4",
"@tailwindcss/oxide-linux-arm64-musl": "4.2.1", "@tailwindcss/oxide-linux-arm64-musl": "4.2.4",
"@tailwindcss/oxide-linux-x64-gnu": "4.2.1", "@tailwindcss/oxide-linux-x64-gnu": "4.2.4",
"@tailwindcss/oxide-linux-x64-musl": "4.2.1", "@tailwindcss/oxide-linux-x64-musl": "4.2.4",
"@tailwindcss/oxide-wasm32-wasi": "4.2.1", "@tailwindcss/oxide-wasm32-wasi": "4.2.4",
"@tailwindcss/oxide-win32-arm64-msvc": "4.2.1", "@tailwindcss/oxide-win32-arm64-msvc": "4.2.4",
"@tailwindcss/oxide-win32-x64-msvc": "4.2.1" "@tailwindcss/oxide-win32-x64-msvc": "4.2.4"
} }
}, },
"node_modules/@tailwindcss/oxide-android-arm64": { "node_modules/@tailwindcss/oxide-android-arm64": {
"version": "4.2.1", "version": "4.2.4",
"resolved": "https://registry.npmjs.org/@tailwindcss/oxide-android-arm64/-/oxide-android-arm64-4.2.1.tgz", "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-android-arm64/-/oxide-android-arm64-4.2.4.tgz",
"integrity": "sha512-eZ7G1Zm5EC8OOKaesIKuw77jw++QJ2lL9N+dDpdQiAB/c/B2wDh0QPFHbkBVrXnwNugvrbJFk1gK2SsVjwWReg==", "integrity": "sha512-e7MOr1SAn9U8KlZzPi1ZXGZHeC5anY36qjNwmZv9pOJ8E4Q6jmD1vyEHkQFmNOIN7twGPEMXRHmitN4zCMN03g==",
"cpu": [ "cpu": [
"arm64" "arm64"
], ],
@@ -1927,9 +1927,9 @@
} }
}, },
"node_modules/@tailwindcss/oxide-darwin-arm64": { "node_modules/@tailwindcss/oxide-darwin-arm64": {
"version": "4.2.1", "version": "4.2.4",
"resolved": "https://registry.npmjs.org/@tailwindcss/oxide-darwin-arm64/-/oxide-darwin-arm64-4.2.1.tgz", "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-darwin-arm64/-/oxide-darwin-arm64-4.2.4.tgz",
"integrity": "sha512-q/LHkOstoJ7pI1J0q6djesLzRvQSIfEto148ppAd+BVQK0JYjQIFSK3JgYZJa+Yzi0DDa52ZsQx2rqytBnf8Hw==", "integrity": "sha512-tSC/Kbqpz/5/o/C2sG7QvOxAKqyd10bq+ypZNf+9Fi2TvbVbv1zNpcEptcsU7DPROaSbVgUXmrzKhurFvo5eDg==",
"cpu": [ "cpu": [
"arm64" "arm64"
], ],
@@ -1944,9 +1944,9 @@
} }
}, },
"node_modules/@tailwindcss/oxide-darwin-x64": { "node_modules/@tailwindcss/oxide-darwin-x64": {
"version": "4.2.1", "version": "4.2.4",
"resolved": "https://registry.npmjs.org/@tailwindcss/oxide-darwin-x64/-/oxide-darwin-x64-4.2.1.tgz", "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-darwin-x64/-/oxide-darwin-x64-4.2.4.tgz",
"integrity": "sha512-/f/ozlaXGY6QLbpvd/kFTro2l18f7dHKpB+ieXz+Cijl4Mt9AI2rTrpq7V+t04nK+j9XBQHnSMdeQRhbGyt6fw==", "integrity": "sha512-yPyUXn3yO/ufR6+Kzv0t4fCg2qNr90jxXc5QqBpjlPNd0NqyDXcmQb/6weunH/MEDXW5dhyEi+agTDiqa3WsGg==",
"cpu": [ "cpu": [
"x64" "x64"
], ],
@@ -1961,9 +1961,9 @@
} }
}, },
"node_modules/@tailwindcss/oxide-freebsd-x64": { "node_modules/@tailwindcss/oxide-freebsd-x64": {
"version": "4.2.1", "version": "4.2.4",
"resolved": "https://registry.npmjs.org/@tailwindcss/oxide-freebsd-x64/-/oxide-freebsd-x64-4.2.1.tgz", "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-freebsd-x64/-/oxide-freebsd-x64-4.2.4.tgz",
"integrity": "sha512-5e/AkgYJT/cpbkys/OU2Ei2jdETCLlifwm7ogMC7/hksI2fC3iiq6OcXwjibcIjPung0kRtR3TxEITkqgn0TcA==", "integrity": "sha512-BoMIB4vMQtZsXdGLVc2z+P9DbETkiopogfWZKbWwM8b/1Vinbs4YcUwo+kM/KeLkX3Ygrf4/PsRndKaYhS8Eiw==",
"cpu": [ "cpu": [
"x64" "x64"
], ],
@@ -1978,9 +1978,9 @@
} }
}, },
"node_modules/@tailwindcss/oxide-linux-arm-gnueabihf": { "node_modules/@tailwindcss/oxide-linux-arm-gnueabihf": {
"version": "4.2.1", "version": "4.2.4",
"resolved": "https://registry.npmjs.org/@tailwindcss/oxide-linux-arm-gnueabihf/-/oxide-linux-arm-gnueabihf-4.2.1.tgz", "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-linux-arm-gnueabihf/-/oxide-linux-arm-gnueabihf-4.2.4.tgz",
"integrity": "sha512-Uny1EcVTTmerCKt/1ZuKTkb0x8ZaiuYucg2/kImO5A5Y/kBz41/+j0gxUZl+hTF3xkWpDmHX+TaWhOtba2Fyuw==", "integrity": "sha512-7pIHBLTHYRAlS7V22JNuTh33yLH4VElwKtB3bwchK/UaKUPpQ0lPQiOWcbm4V3WP2I6fNIJ23vABIvoy2izdwA==",
"cpu": [ "cpu": [
"arm" "arm"
], ],
@@ -1995,9 +1995,9 @@
} }
}, },
"node_modules/@tailwindcss/oxide-linux-arm64-gnu": { "node_modules/@tailwindcss/oxide-linux-arm64-gnu": {
"version": "4.2.1", "version": "4.2.4",
"resolved": "https://registry.npmjs.org/@tailwindcss/oxide-linux-arm64-gnu/-/oxide-linux-arm64-gnu-4.2.1.tgz", "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-linux-arm64-gnu/-/oxide-linux-arm64-gnu-4.2.4.tgz",
"integrity": "sha512-CTrwomI+c7n6aSSQlsPL0roRiNMDQ/YzMD9EjcR+H4f0I1SQ8QqIuPnsVp7QgMkC1Qi8rtkekLkOFjo7OlEFRQ==", "integrity": "sha512-+E4wxJ0ZGOzSH325reXTWB48l42i93kQqMvDyz5gqfRzRZ7faNhnmvlV4EPGJU3QJM/3Ab5jhJ5pCRUsKn6OQw==",
"cpu": [ "cpu": [
"arm64" "arm64"
], ],
@@ -2012,9 +2012,9 @@
} }
}, },
"node_modules/@tailwindcss/oxide-linux-arm64-musl": { "node_modules/@tailwindcss/oxide-linux-arm64-musl": {
"version": "4.2.1", "version": "4.2.4",
"resolved": "https://registry.npmjs.org/@tailwindcss/oxide-linux-arm64-musl/-/oxide-linux-arm64-musl-4.2.1.tgz", "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-linux-arm64-musl/-/oxide-linux-arm64-musl-4.2.4.tgz",
"integrity": "sha512-WZA0CHRL/SP1TRbA5mp9htsppSEkWuQ4KsSUumYQnyl8ZdT39ntwqmz4IUHGN6p4XdSlYfJwM4rRzZLShHsGAQ==", "integrity": "sha512-bBADEGAbo4ASnppIziaQJelekCxdMaxisrk+fB7Thit72IBnALp9K6ffA2G4ruj90G9XRS2VQ6q2bCKbfFV82g==",
"cpu": [ "cpu": [
"arm64" "arm64"
], ],
@@ -2029,9 +2029,9 @@
} }
}, },
"node_modules/@tailwindcss/oxide-linux-x64-gnu": { "node_modules/@tailwindcss/oxide-linux-x64-gnu": {
"version": "4.2.1", "version": "4.2.4",
"resolved": "https://registry.npmjs.org/@tailwindcss/oxide-linux-x64-gnu/-/oxide-linux-x64-gnu-4.2.1.tgz", "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-linux-x64-gnu/-/oxide-linux-x64-gnu-4.2.4.tgz",
"integrity": "sha512-qMFzxI2YlBOLW5PhblzuSWlWfwLHaneBE0xHzLrBgNtqN6mWfs+qYbhryGSXQjFYB1Dzf5w+LN5qbUTPhW7Y5g==", "integrity": "sha512-7Mx25E4WTfnht0TVRTyC00j3i0M+EeFe7wguMDTlX4mRxafznw0CA8WJkFjWYH5BlgELd1kSjuU2JiPnNZbJDA==",
"cpu": [ "cpu": [
"x64" "x64"
], ],
@@ -2046,9 +2046,9 @@
} }
}, },
"node_modules/@tailwindcss/oxide-linux-x64-musl": { "node_modules/@tailwindcss/oxide-linux-x64-musl": {
"version": "4.2.1", "version": "4.2.4",
"resolved": "https://registry.npmjs.org/@tailwindcss/oxide-linux-x64-musl/-/oxide-linux-x64-musl-4.2.1.tgz", "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-linux-x64-musl/-/oxide-linux-x64-musl-4.2.4.tgz",
"integrity": "sha512-5r1X2FKnCMUPlXTWRYpHdPYUY6a1Ar/t7P24OuiEdEOmms5lyqjDRvVY1yy9Rmioh+AunQ0rWiOTPE8F9A3v5g==", "integrity": "sha512-2wwJRF7nyhOR0hhHoChc04xngV3iS+akccHTGtz965FwF0up4b2lOdo6kI1EbDaEXKgvcrFBYcYQQ/rrnWFVfA==",
"cpu": [ "cpu": [
"x64" "x64"
], ],
@@ -2063,9 +2063,9 @@
} }
}, },
"node_modules/@tailwindcss/oxide-wasm32-wasi": { "node_modules/@tailwindcss/oxide-wasm32-wasi": {
"version": "4.2.1", "version": "4.2.4",
"resolved": "https://registry.npmjs.org/@tailwindcss/oxide-wasm32-wasi/-/oxide-wasm32-wasi-4.2.1.tgz", "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-wasm32-wasi/-/oxide-wasm32-wasi-4.2.4.tgz",
"integrity": "sha512-MGFB5cVPvshR85MTJkEvqDUnuNoysrsRxd6vnk1Lf2tbiqNlXpHYZqkqOQalydienEWOHHFyyuTSYRsLfxFJ2Q==", "integrity": "sha512-FQsqApeor8Fo6gUEklzmaa9994orJZZDBAlQpK2Mq+DslRKFJeD6AjHpBQ0kZFQohVr8o85PPh8eOy86VlSCmw==",
"bundleDependencies": [ "bundleDependencies": [
"@napi-rs/wasm-runtime", "@napi-rs/wasm-runtime",
"@emnapi/core", "@emnapi/core",
@@ -2157,9 +2157,9 @@
"optional": true "optional": true
}, },
"node_modules/@tailwindcss/oxide-win32-arm64-msvc": { "node_modules/@tailwindcss/oxide-win32-arm64-msvc": {
"version": "4.2.1", "version": "4.2.4",
"resolved": "https://registry.npmjs.org/@tailwindcss/oxide-win32-arm64-msvc/-/oxide-win32-arm64-msvc-4.2.1.tgz", "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-win32-arm64-msvc/-/oxide-win32-arm64-msvc-4.2.4.tgz",
"integrity": "sha512-YlUEHRHBGnCMh4Nj4GnqQyBtsshUPdiNroZj8VPkvTZSoHsilRCwXcVKnG9kyi0ZFAS/3u+qKHBdDc81SADTRA==", "integrity": "sha512-L9BXqxC4ToVgwMFqj3pmZRqyHEztulpUJzCxUtLjobMCzTPsGt1Fa9enKbOpY2iIyVtaHNeNvAK8ERP/64sqGQ==",
"cpu": [ "cpu": [
"arm64" "arm64"
], ],
@@ -2174,9 +2174,9 @@
} }
}, },
"node_modules/@tailwindcss/oxide-win32-x64-msvc": { "node_modules/@tailwindcss/oxide-win32-x64-msvc": {
"version": "4.2.1", "version": "4.2.4",
"resolved": "https://registry.npmjs.org/@tailwindcss/oxide-win32-x64-msvc/-/oxide-win32-x64-msvc-4.2.1.tgz", "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-win32-x64-msvc/-/oxide-win32-x64-msvc-4.2.4.tgz",
"integrity": "sha512-rbO34G5sMWWyrN/idLeVxAZgAKWrn5LiR3/I90Q9MkA67s6T1oB0xtTe+0heoBvHSpbU9Mk7i6uwJnpo4u21XQ==", "integrity": "sha512-ESlKG0EpVJQwRjXDDa9rLvhEAh0mhP1sF7sap9dNZT0yyl9SAG6T7gdP09EH0vIv0UNTlo6jPWyujD6559fZvw==",
"cpu": [ "cpu": [
"x64" "x64"
], ],
@@ -2191,17 +2191,17 @@
} }
}, },
"node_modules/@tailwindcss/postcss": { "node_modules/@tailwindcss/postcss": {
"version": "4.2.1", "version": "4.2.4",
"resolved": "https://registry.npmjs.org/@tailwindcss/postcss/-/postcss-4.2.1.tgz", "resolved": "https://registry.npmjs.org/@tailwindcss/postcss/-/postcss-4.2.4.tgz",
"integrity": "sha512-OEwGIBnXnj7zJeonOh6ZG9woofIjGrd2BORfvE5p9USYKDCZoQmfqLcfNiRWoJlRWLdNPn2IgVZuWAOM4iTYMw==", "integrity": "sha512-wgAVj6nUWAolAu8YFvzT2cTBIElWHkjZwFYovF+xsqKsW2ADxM/X2opxj5NsF/qVccAOjRNe8X2IdPzMsWyHTg==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@alloc/quick-lru": "^5.2.0", "@alloc/quick-lru": "^5.2.0",
"@tailwindcss/node": "4.2.1", "@tailwindcss/node": "4.2.4",
"@tailwindcss/oxide": "4.2.1", "@tailwindcss/oxide": "4.2.4",
"postcss": "^8.5.6", "postcss": "^8.5.6",
"tailwindcss": "4.2.1" "tailwindcss": "4.2.4"
} }
}, },
"node_modules/@tauri-apps/api": { "node_modules/@tauri-apps/api": {
@@ -4198,14 +4198,14 @@
"license": "MIT" "license": "MIT"
}, },
"node_modules/enhanced-resolve": { "node_modules/enhanced-resolve": {
"version": "5.19.0", "version": "5.21.0",
"resolved": "https://registry.npmjs.org/enhanced-resolve/-/enhanced-resolve-5.19.0.tgz", "resolved": "https://registry.npmjs.org/enhanced-resolve/-/enhanced-resolve-5.21.0.tgz",
"integrity": "sha512-phv3E1Xl4tQOShqSte26C7Fl84EwUdZsyOuSSk9qtAGyyQs2s3jJzComh+Abf4g187lUUAvH+H26omrqia2aGg==", "integrity": "sha512-otxSQPw4lkOZWkHpB3zaEQs6gWYEsmX4xQF68ElXC/TWvGxGMSGOvoNbaLXm6/cS/fSfHtsEdw90y20PCd+sCA==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"graceful-fs": "^4.2.4", "graceful-fs": "^4.2.4",
"tapable": "^2.3.0" "tapable": "^2.3.3"
}, },
"engines": { "engines": {
"node": ">=10.13.0" "node": ">=10.13.0"
@@ -4492,13 +4492,13 @@
} }
}, },
"node_modules/eslint-config-next": { "node_modules/eslint-config-next": {
"version": "16.1.6", "version": "16.2.4",
"resolved": "https://registry.npmjs.org/eslint-config-next/-/eslint-config-next-16.1.6.tgz", "resolved": "https://registry.npmjs.org/eslint-config-next/-/eslint-config-next-16.2.4.tgz",
"integrity": "sha512-vKq40io2B0XtkkNDYyleATwblNt8xuh3FWp8SpSz3pt7P01OkBFlKsJZ2mWt5WsCySlDQLckb1zMY9yE9Qy0LA==", "integrity": "sha512-A6ekXYFj/YQxBPMl45g3e+U8zJo+X2+ZQwcz34pPKjpc/3S4roBA2Rd9xWB4FKuSxhofo1/95WjzmUY+wHrOhg==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@next/eslint-plugin-next": "16.1.6", "@next/eslint-plugin-next": "16.2.4",
"eslint-import-resolver-node": "^0.3.6", "eslint-import-resolver-node": "^0.3.6",
"eslint-import-resolver-typescript": "^3.5.2", "eslint-import-resolver-typescript": "^3.5.2",
"eslint-plugin-import": "^2.32.0", "eslint-plugin-import": "^2.32.0",
@@ -6338,9 +6338,9 @@
} }
}, },
"node_modules/lightningcss": { "node_modules/lightningcss": {
"version": "1.31.1", "version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss/-/lightningcss-1.31.1.tgz", "resolved": "https://registry.npmjs.org/lightningcss/-/lightningcss-1.32.0.tgz",
"integrity": "sha512-l51N2r93WmGUye3WuFoN5k10zyvrVs0qfKBhyC5ogUQ6Ew6JUSswh78mbSO+IU3nTWsyOArqPCcShdQSadghBQ==", "integrity": "sha512-NXYBzinNrblfraPGyrbPoD19C1h9lfI/1mzgWYvXUTe414Gz/X1FD2XBZSZM7rRTrMA8JL3OtAaGifrIKhQ5yQ==",
"dev": true, "dev": true,
"license": "MPL-2.0", "license": "MPL-2.0",
"dependencies": { "dependencies": {
@@ -6354,23 +6354,23 @@
"url": "https://opencollective.com/parcel" "url": "https://opencollective.com/parcel"
}, },
"optionalDependencies": { "optionalDependencies": {
"lightningcss-android-arm64": "1.31.1", "lightningcss-android-arm64": "1.32.0",
"lightningcss-darwin-arm64": "1.31.1", "lightningcss-darwin-arm64": "1.32.0",
"lightningcss-darwin-x64": "1.31.1", "lightningcss-darwin-x64": "1.32.0",
"lightningcss-freebsd-x64": "1.31.1", "lightningcss-freebsd-x64": "1.32.0",
"lightningcss-linux-arm-gnueabihf": "1.31.1", "lightningcss-linux-arm-gnueabihf": "1.32.0",
"lightningcss-linux-arm64-gnu": "1.31.1", "lightningcss-linux-arm64-gnu": "1.32.0",
"lightningcss-linux-arm64-musl": "1.31.1", "lightningcss-linux-arm64-musl": "1.32.0",
"lightningcss-linux-x64-gnu": "1.31.1", "lightningcss-linux-x64-gnu": "1.32.0",
"lightningcss-linux-x64-musl": "1.31.1", "lightningcss-linux-x64-musl": "1.32.0",
"lightningcss-win32-arm64-msvc": "1.31.1", "lightningcss-win32-arm64-msvc": "1.32.0",
"lightningcss-win32-x64-msvc": "1.31.1" "lightningcss-win32-x64-msvc": "1.32.0"
} }
}, },
"node_modules/lightningcss-android-arm64": { "node_modules/lightningcss-android-arm64": {
"version": "1.31.1", "version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-android-arm64/-/lightningcss-android-arm64-1.31.1.tgz", "resolved": "https://registry.npmjs.org/lightningcss-android-arm64/-/lightningcss-android-arm64-1.32.0.tgz",
"integrity": "sha512-HXJF3x8w9nQ4jbXRiNppBCqeZPIAfUo8zE/kOEGbW5NZvGc/K7nMxbhIr+YlFlHW5mpbg/YFPdbnCh1wAXCKFg==", "integrity": "sha512-YK7/ClTt4kAK0vo6w3X+Pnm0D2cf2vPHbhOXdoNti1Ga0al1P4TBZhwjATvjNwLEBCnKvjJc2jQgHXH0NEwlAg==",
"cpu": [ "cpu": [
"arm64" "arm64"
], ],
@@ -6389,9 +6389,9 @@
} }
}, },
"node_modules/lightningcss-darwin-arm64": { "node_modules/lightningcss-darwin-arm64": {
"version": "1.31.1", "version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-darwin-arm64/-/lightningcss-darwin-arm64-1.31.1.tgz", "resolved": "https://registry.npmjs.org/lightningcss-darwin-arm64/-/lightningcss-darwin-arm64-1.32.0.tgz",
"integrity": "sha512-02uTEqf3vIfNMq3h/z2cJfcOXnQ0GRwQrkmPafhueLb2h7mqEidiCzkE4gBMEH65abHRiQvhdcQ+aP0D0g67sg==", "integrity": "sha512-RzeG9Ju5bag2Bv1/lwlVJvBE3q6TtXskdZLLCyfg5pt+HLz9BqlICO7LZM7VHNTTn/5PRhHFBSjk5lc4cmscPQ==",
"cpu": [ "cpu": [
"arm64" "arm64"
], ],
@@ -6410,9 +6410,9 @@
} }
}, },
"node_modules/lightningcss-darwin-x64": { "node_modules/lightningcss-darwin-x64": {
"version": "1.31.1", "version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-darwin-x64/-/lightningcss-darwin-x64-1.31.1.tgz", "resolved": "https://registry.npmjs.org/lightningcss-darwin-x64/-/lightningcss-darwin-x64-1.32.0.tgz",
"integrity": "sha512-1ObhyoCY+tGxtsz1lSx5NXCj3nirk0Y0kB/g8B8DT+sSx4G9djitg9ejFnjb3gJNWo7qXH4DIy2SUHvpoFwfTA==", "integrity": "sha512-U+QsBp2m/s2wqpUYT/6wnlagdZbtZdndSmut/NJqlCcMLTWp5muCrID+K5UJ6jqD2BFshejCYXniPDbNh73V8w==",
"cpu": [ "cpu": [
"x64" "x64"
], ],
@@ -6431,9 +6431,9 @@
} }
}, },
"node_modules/lightningcss-freebsd-x64": { "node_modules/lightningcss-freebsd-x64": {
"version": "1.31.1", "version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-freebsd-x64/-/lightningcss-freebsd-x64-1.31.1.tgz", "resolved": "https://registry.npmjs.org/lightningcss-freebsd-x64/-/lightningcss-freebsd-x64-1.32.0.tgz",
"integrity": "sha512-1RINmQKAItO6ISxYgPwszQE1BrsVU5aB45ho6O42mu96UiZBxEXsuQ7cJW4zs4CEodPUioj/QrXW1r9pLUM74A==", "integrity": "sha512-JCTigedEksZk3tHTTthnMdVfGf61Fky8Ji2E4YjUTEQX14xiy/lTzXnu1vwiZe3bYe0q+SpsSH/CTeDXK6WHig==",
"cpu": [ "cpu": [
"x64" "x64"
], ],
@@ -6452,9 +6452,9 @@
} }
}, },
"node_modules/lightningcss-linux-arm-gnueabihf": { "node_modules/lightningcss-linux-arm-gnueabihf": {
"version": "1.31.1", "version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-linux-arm-gnueabihf/-/lightningcss-linux-arm-gnueabihf-1.31.1.tgz", "resolved": "https://registry.npmjs.org/lightningcss-linux-arm-gnueabihf/-/lightningcss-linux-arm-gnueabihf-1.32.0.tgz",
"integrity": "sha512-OOCm2//MZJ87CdDK62rZIu+aw9gBv4azMJuA8/KB74wmfS3lnC4yoPHm0uXZ/dvNNHmnZnB8XLAZzObeG0nS1g==", "integrity": "sha512-x6rnnpRa2GL0zQOkt6rts3YDPzduLpWvwAF6EMhXFVZXD4tPrBkEFqzGowzCsIWsPjqSK+tyNEODUBXeeVHSkw==",
"cpu": [ "cpu": [
"arm" "arm"
], ],
@@ -6473,9 +6473,9 @@
} }
}, },
"node_modules/lightningcss-linux-arm64-gnu": { "node_modules/lightningcss-linux-arm64-gnu": {
"version": "1.31.1", "version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-linux-arm64-gnu/-/lightningcss-linux-arm64-gnu-1.31.1.tgz", "resolved": "https://registry.npmjs.org/lightningcss-linux-arm64-gnu/-/lightningcss-linux-arm64-gnu-1.32.0.tgz",
"integrity": "sha512-WKyLWztD71rTnou4xAD5kQT+982wvca7E6QoLpoawZ1gP9JM0GJj4Tp5jMUh9B3AitHbRZ2/H3W5xQmdEOUlLg==", "integrity": "sha512-0nnMyoyOLRJXfbMOilaSRcLH3Jw5z9HDNGfT/gwCPgaDjnx0i8w7vBzFLFR1f6CMLKF8gVbebmkUN3fa/kQJpQ==",
"cpu": [ "cpu": [
"arm64" "arm64"
], ],
@@ -6494,9 +6494,9 @@
} }
}, },
"node_modules/lightningcss-linux-arm64-musl": { "node_modules/lightningcss-linux-arm64-musl": {
"version": "1.31.1", "version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-linux-arm64-musl/-/lightningcss-linux-arm64-musl-1.31.1.tgz", "resolved": "https://registry.npmjs.org/lightningcss-linux-arm64-musl/-/lightningcss-linux-arm64-musl-1.32.0.tgz",
"integrity": "sha512-mVZ7Pg2zIbe3XlNbZJdjs86YViQFoJSpc41CbVmKBPiGmC4YrfeOyz65ms2qpAobVd7WQsbW4PdsSJEMymyIMg==", "integrity": "sha512-UpQkoenr4UJEzgVIYpI80lDFvRmPVg6oqboNHfoH4CQIfNA+HOrZ7Mo7KZP02dC6LjghPQJeBsvXhJod/wnIBg==",
"cpu": [ "cpu": [
"arm64" "arm64"
], ],
@@ -6515,9 +6515,9 @@
} }
}, },
"node_modules/lightningcss-linux-x64-gnu": { "node_modules/lightningcss-linux-x64-gnu": {
"version": "1.31.1", "version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-linux-x64-gnu/-/lightningcss-linux-x64-gnu-1.31.1.tgz", "resolved": "https://registry.npmjs.org/lightningcss-linux-x64-gnu/-/lightningcss-linux-x64-gnu-1.32.0.tgz",
"integrity": "sha512-xGlFWRMl+0KvUhgySdIaReQdB4FNudfUTARn7q0hh/V67PVGCs3ADFjw+6++kG1RNd0zdGRlEKa+T13/tQjPMA==", "integrity": "sha512-V7Qr52IhZmdKPVr+Vtw8o+WLsQJYCTd8loIfpDaMRWGUZfBOYEJeyJIkqGIDMZPwPx24pUMfwSxxI8phr/MbOA==",
"cpu": [ "cpu": [
"x64" "x64"
], ],
@@ -6536,9 +6536,9 @@
} }
}, },
"node_modules/lightningcss-linux-x64-musl": { "node_modules/lightningcss-linux-x64-musl": {
"version": "1.31.1", "version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-linux-x64-musl/-/lightningcss-linux-x64-musl-1.31.1.tgz", "resolved": "https://registry.npmjs.org/lightningcss-linux-x64-musl/-/lightningcss-linux-x64-musl-1.32.0.tgz",
"integrity": "sha512-eowF8PrKHw9LpoZii5tdZwnBcYDxRw2rRCyvAXLi34iyeYfqCQNA9rmUM0ce62NlPhCvof1+9ivRaTY6pSKDaA==", "integrity": "sha512-bYcLp+Vb0awsiXg/80uCRezCYHNg1/l3mt0gzHnWV9XP1W5sKa5/TCdGWaR/zBM2PeF/HbsQv/j2URNOiVuxWg==",
"cpu": [ "cpu": [
"x64" "x64"
], ],
@@ -6557,9 +6557,9 @@
} }
}, },
"node_modules/lightningcss-win32-arm64-msvc": { "node_modules/lightningcss-win32-arm64-msvc": {
"version": "1.31.1", "version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-win32-arm64-msvc/-/lightningcss-win32-arm64-msvc-1.31.1.tgz", "resolved": "https://registry.npmjs.org/lightningcss-win32-arm64-msvc/-/lightningcss-win32-arm64-msvc-1.32.0.tgz",
"integrity": "sha512-aJReEbSEQzx1uBlQizAOBSjcmr9dCdL3XuC/6HLXAxmtErsj2ICo5yYggg1qOODQMtnjNQv2UHb9NpOuFtYe4w==", "integrity": "sha512-8SbC8BR40pS6baCM8sbtYDSwEVQd4JlFTOlaD3gWGHfThTcABnNDBda6eTZeqbofalIJhFx0qKzgHJmcPTnGdw==",
"cpu": [ "cpu": [
"arm64" "arm64"
], ],
@@ -6578,9 +6578,9 @@
} }
}, },
"node_modules/lightningcss-win32-x64-msvc": { "node_modules/lightningcss-win32-x64-msvc": {
"version": "1.31.1", "version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-win32-x64-msvc/-/lightningcss-win32-x64-msvc-1.31.1.tgz", "resolved": "https://registry.npmjs.org/lightningcss-win32-x64-msvc/-/lightningcss-win32-x64-msvc-1.32.0.tgz",
"integrity": "sha512-I9aiFrbd7oYHwlnQDqr1Roz+fTz61oDDJX7n9tYF9FJymH1cIN1DtKw3iYt6b8WZgEjoNwVSncwF4wx/ZedMhw==", "integrity": "sha512-Amq9B/SoZYdDi1kFrojnoqPLxYhQ4Wo5XiL8EVJrVsB8ARoC1PWW6VGtT0WKCemjy8aC+louJnjS7U18x3b06Q==",
"cpu": [ "cpu": [
"x64" "x64"
], ],
@@ -7342,9 +7342,9 @@
"license": "ISC" "license": "ISC"
}, },
"node_modules/picomatch": { "node_modules/picomatch": {
"version": "2.3.1", "version": "2.3.2",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.1.tgz", "resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.2.tgz",
"integrity": "sha512-JU3teHTNjmE2VCGFzuY8EXzCDVwEqB2a8fsIvwaStHhAWJEeVd1o1QD80CU6+ZdEXXSLbSsuLwJjkCBWqRQUVA==", "integrity": "sha512-V7+vQEJ06Z+c5tSye8S+nHUfI51xoXIXjHQ99cQtKUkQqqO1kO/KCJUfZXuB47h/YBlDhah2H3hdUGXn8ie0oA==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"engines": { "engines": {
@@ -7419,9 +7419,9 @@
} }
}, },
"node_modules/prettier": { "node_modules/prettier": {
"version": "3.8.1", "version": "3.8.3",
"resolved": "https://registry.npmjs.org/prettier/-/prettier-3.8.1.tgz", "resolved": "https://registry.npmjs.org/prettier/-/prettier-3.8.3.tgz",
"integrity": "sha512-UOnG6LftzbdaHZcKoPFtOcCKztrQ57WkHDeRD9t/PTQtmT0NHSeWWepj6pS0z/N7+08BHFDQVUrfmfMRcZwbMg==", "integrity": "sha512-7igPTM53cGHMW8xWuVTydi2KO233VFiTNyF5hLJqpilHfmn8C8gPf+PS7dUT64YcXFbiMGZxS9pCSxL/Dxm/Jw==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"bin": { "bin": {
@@ -8632,16 +8632,16 @@
"license": "MIT" "license": "MIT"
}, },
"node_modules/tailwindcss": { "node_modules/tailwindcss": {
"version": "4.2.1", "version": "4.2.4",
"resolved": "https://registry.npmjs.org/tailwindcss/-/tailwindcss-4.2.1.tgz", "resolved": "https://registry.npmjs.org/tailwindcss/-/tailwindcss-4.2.4.tgz",
"integrity": "sha512-/tBrSQ36vCleJkAOsy9kbNTgaxvGbyOamC30PRePTQe/o1MFwEKHQk4Cn7BNGaPtjp+PuUrByJehM1hgxfq4sw==", "integrity": "sha512-HhKppgO81FQof5m6TEnuBWCZGgfRAWbaeOaGT00KOy/Pf/j6oUihdvBpA7ltCeAvZpFhW3j0PTclkxsd4IXYDA==",
"dev": true, "dev": true,
"license": "MIT" "license": "MIT"
}, },
"node_modules/tapable": { "node_modules/tapable": {
"version": "2.3.0", "version": "2.3.3",
"resolved": "https://registry.npmjs.org/tapable/-/tapable-2.3.0.tgz", "resolved": "https://registry.npmjs.org/tapable/-/tapable-2.3.3.tgz",
"integrity": "sha512-g9ljZiwki/LfxmQADO3dEY1CbpmXT5Hm2fJ+QaGKwSXUylMybePR7/67YW7jOrrvjEgL1Fmz5kzyAjWVWLlucg==", "integrity": "sha512-uxc/zpqFg6x7C8vOE7lh6Lbda8eEL9zmVm/PLeTPBRhh1xCgdWaQ+J1CUieGpIfm2HdtsUpRv+HshiasBMcc6A==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"engines": { "engines": {
@@ -9197,267 +9197,6 @@
} }
} }
}, },
"node_modules/vite/node_modules/lightningcss": {
"version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss/-/lightningcss-1.32.0.tgz",
"integrity": "sha512-NXYBzinNrblfraPGyrbPoD19C1h9lfI/1mzgWYvXUTe414Gz/X1FD2XBZSZM7rRTrMA8JL3OtAaGifrIKhQ5yQ==",
"dev": true,
"license": "MPL-2.0",
"dependencies": {
"detect-libc": "^2.0.3"
},
"engines": {
"node": ">= 12.0.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/parcel"
},
"optionalDependencies": {
"lightningcss-android-arm64": "1.32.0",
"lightningcss-darwin-arm64": "1.32.0",
"lightningcss-darwin-x64": "1.32.0",
"lightningcss-freebsd-x64": "1.32.0",
"lightningcss-linux-arm-gnueabihf": "1.32.0",
"lightningcss-linux-arm64-gnu": "1.32.0",
"lightningcss-linux-arm64-musl": "1.32.0",
"lightningcss-linux-x64-gnu": "1.32.0",
"lightningcss-linux-x64-musl": "1.32.0",
"lightningcss-win32-arm64-msvc": "1.32.0",
"lightningcss-win32-x64-msvc": "1.32.0"
}
},
"node_modules/vite/node_modules/lightningcss-android-arm64": {
"version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-android-arm64/-/lightningcss-android-arm64-1.32.0.tgz",
"integrity": "sha512-YK7/ClTt4kAK0vo6w3X+Pnm0D2cf2vPHbhOXdoNti1Ga0al1P4TBZhwjATvjNwLEBCnKvjJc2jQgHXH0NEwlAg==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MPL-2.0",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">= 12.0.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/parcel"
}
},
"node_modules/vite/node_modules/lightningcss-darwin-arm64": {
"version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-darwin-arm64/-/lightningcss-darwin-arm64-1.32.0.tgz",
"integrity": "sha512-RzeG9Ju5bag2Bv1/lwlVJvBE3q6TtXskdZLLCyfg5pt+HLz9BqlICO7LZM7VHNTTn/5PRhHFBSjk5lc4cmscPQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MPL-2.0",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">= 12.0.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/parcel"
}
},
"node_modules/vite/node_modules/lightningcss-darwin-x64": {
"version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-darwin-x64/-/lightningcss-darwin-x64-1.32.0.tgz",
"integrity": "sha512-U+QsBp2m/s2wqpUYT/6wnlagdZbtZdndSmut/NJqlCcMLTWp5muCrID+K5UJ6jqD2BFshejCYXniPDbNh73V8w==",
"cpu": [
"x64"
],
"dev": true,
"license": "MPL-2.0",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">= 12.0.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/parcel"
}
},
"node_modules/vite/node_modules/lightningcss-freebsd-x64": {
"version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-freebsd-x64/-/lightningcss-freebsd-x64-1.32.0.tgz",
"integrity": "sha512-JCTigedEksZk3tHTTthnMdVfGf61Fky8Ji2E4YjUTEQX14xiy/lTzXnu1vwiZe3bYe0q+SpsSH/CTeDXK6WHig==",
"cpu": [
"x64"
],
"dev": true,
"license": "MPL-2.0",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">= 12.0.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/parcel"
}
},
"node_modules/vite/node_modules/lightningcss-linux-arm-gnueabihf": {
"version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-linux-arm-gnueabihf/-/lightningcss-linux-arm-gnueabihf-1.32.0.tgz",
"integrity": "sha512-x6rnnpRa2GL0zQOkt6rts3YDPzduLpWvwAF6EMhXFVZXD4tPrBkEFqzGowzCsIWsPjqSK+tyNEODUBXeeVHSkw==",
"cpu": [
"arm"
],
"dev": true,
"license": "MPL-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 12.0.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/parcel"
}
},
"node_modules/vite/node_modules/lightningcss-linux-arm64-gnu": {
"version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-linux-arm64-gnu/-/lightningcss-linux-arm64-gnu-1.32.0.tgz",
"integrity": "sha512-0nnMyoyOLRJXfbMOilaSRcLH3Jw5z9HDNGfT/gwCPgaDjnx0i8w7vBzFLFR1f6CMLKF8gVbebmkUN3fa/kQJpQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MPL-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 12.0.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/parcel"
}
},
"node_modules/vite/node_modules/lightningcss-linux-arm64-musl": {
"version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-linux-arm64-musl/-/lightningcss-linux-arm64-musl-1.32.0.tgz",
"integrity": "sha512-UpQkoenr4UJEzgVIYpI80lDFvRmPVg6oqboNHfoH4CQIfNA+HOrZ7Mo7KZP02dC6LjghPQJeBsvXhJod/wnIBg==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MPL-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 12.0.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/parcel"
}
},
"node_modules/vite/node_modules/lightningcss-linux-x64-gnu": {
"version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-linux-x64-gnu/-/lightningcss-linux-x64-gnu-1.32.0.tgz",
"integrity": "sha512-V7Qr52IhZmdKPVr+Vtw8o+WLsQJYCTd8loIfpDaMRWGUZfBOYEJeyJIkqGIDMZPwPx24pUMfwSxxI8phr/MbOA==",
"cpu": [
"x64"
],
"dev": true,
"license": "MPL-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 12.0.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/parcel"
}
},
"node_modules/vite/node_modules/lightningcss-linux-x64-musl": {
"version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-linux-x64-musl/-/lightningcss-linux-x64-musl-1.32.0.tgz",
"integrity": "sha512-bYcLp+Vb0awsiXg/80uCRezCYHNg1/l3mt0gzHnWV9XP1W5sKa5/TCdGWaR/zBM2PeF/HbsQv/j2URNOiVuxWg==",
"cpu": [
"x64"
],
"dev": true,
"license": "MPL-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 12.0.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/parcel"
}
},
"node_modules/vite/node_modules/lightningcss-win32-arm64-msvc": {
"version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-win32-arm64-msvc/-/lightningcss-win32-arm64-msvc-1.32.0.tgz",
"integrity": "sha512-8SbC8BR40pS6baCM8sbtYDSwEVQd4JlFTOlaD3gWGHfThTcABnNDBda6eTZeqbofalIJhFx0qKzgHJmcPTnGdw==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MPL-2.0",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">= 12.0.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/parcel"
}
},
"node_modules/vite/node_modules/lightningcss-win32-x64-msvc": {
"version": "1.32.0",
"resolved": "https://registry.npmjs.org/lightningcss-win32-x64-msvc/-/lightningcss-win32-x64-msvc-1.32.0.tgz",
"integrity": "sha512-Amq9B/SoZYdDi1kFrojnoqPLxYhQ4Wo5XiL8EVJrVsB8ARoC1PWW6VGtT0WKCemjy8aC+louJnjS7U18x3b06Q==",
"cpu": [
"x64"
],
"dev": true,
"license": "MPL-2.0",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">= 12.0.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/parcel"
}
},
"node_modules/vite/node_modules/picomatch": { "node_modules/vite/node_modules/picomatch": {
"version": "4.0.3", "version": "4.0.3",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.3.tgz", "resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.3.tgz",
+3 -3
View File
@@ -1,6 +1,6 @@
{ {
"name": "frontend", "name": "frontend",
"version": "0.9.7", "version": "0.9.75",
"private": true, "private": true,
"scripts": { "scripts": {
"dev": "node scripts/dev-all.cjs", "dev": "node scripts/dev-all.cjs",
@@ -45,9 +45,9 @@
"@vitest/coverage-v8": "^4.1.0", "@vitest/coverage-v8": "^4.1.0",
"concurrently": "^9.2.1", "concurrently": "^9.2.1",
"eslint": "^9", "eslint": "^9",
"eslint-config-next": "16.1.6", "eslint-config-next": "16.2.4",
"jsdom": "^28.1.0", "jsdom": "^28.1.0",
"prettier": "^3.3.3", "prettier": "^3.8.3",
"tailwindcss": "^4", "tailwindcss": "^4",
"typescript": "^5", "typescript": "^5",
"vitest": "^4.1.0" "vitest": "^4.1.0"
@@ -2,8 +2,8 @@
* Phase 5F-A: CSP nonce plumbing tests. * Phase 5F-A: CSP nonce plumbing tests.
* *
* Validates: * Validates:
* 1. Nonce appears in document CSP header * 1. Document CSP remains hydration-safe for the Next.js runtime
* 2. Nonce differs across repeated requests * 2. CSP is deterministic across repeated requests
* 3. next.config.ts no longer owns a static CSP header * 3. next.config.ts no longer owns a static CSP header
* 4. Middleware does not break API/static routes (matcher exclusion) * 4. Middleware does not break API/static routes (matcher exclusion)
* 5. Google Fonts domains are preserved in CSP * 5. Google Fonts domains are preserved in CSP
@@ -41,58 +41,46 @@ function matcherExcludes(path: string): boolean {
} }
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// 1. Nonce appears in document CSP header // 1. Document CSP remains hydration-safe
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
describe('nonce in CSP header', () => { describe('hydration-safe CSP header', () => {
it('CSP header contains a nonce-<value> token in script-src', () => { it('CSP header does not put nonce tokens in script-src', () => {
const csp = getCsp(); const csp = getCsp();
expect(csp).toMatch(/'nonce-[A-Za-z0-9+/=]+'/) ; expect(csp).not.toMatch(/'nonce-[A-Za-z0-9+/=]+'/);
}); });
it('nonce value is a base64-encoded UUID', () => { it('script-src keeps the inline compatibility fallback required by Next hydration', () => {
const csp = getCsp(); const csp = getCsp();
const match = csp.match(/'nonce-([A-Za-z0-9+/=]+)'/); expect(csp).toMatch(/script-src [^;]*'unsafe-inline'/);
expect(match).not.toBeNull();
const decoded = Buffer.from(match![1], 'base64').toString();
// crypto.randomUUID() produces 8-4-4-4-12 hex with dashes
expect(decoded).toMatch(/^[0-9a-f]{8}-[0-9a-f]{4}-/);
}); });
it('x-nonce request header is set on the response', () => { it('middleware still returns a CSP header for document requests', () => {
const res = callMiddleware(); const csp = getCsp();
// NextResponse.next({ request: { headers } }) merges into request headers. expect(csp).toContain("default-src 'self'");
// The CSP nonce in the header must match the one forwarded to server components. expect(csp).toContain("script-src 'self'");
const csp = res.headers.get('Content-Security-Policy') ?? '';
const nonceInCsp = csp.match(/'nonce-([A-Za-z0-9+/=]+)'/)?.[1];
expect(nonceInCsp).toBeTruthy();
}); });
}); });
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// 2. Nonce differs across repeated requests // 2. CSP is deterministic across repeated requests
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
describe('nonce uniqueness', () => { describe('CSP stability', () => {
it('two sequential requests produce different nonces', () => { it('two sequential requests produce the same document CSP', () => {
const csp1 = getCsp(); const csp1 = getCsp();
const csp2 = getCsp(); const csp2 = getCsp();
const nonce1 = csp1.match(/'nonce-([A-Za-z0-9+/=]+)'/)?.[1]; expect(csp1).toBe(csp2);
const nonce2 = csp2.match(/'nonce-([A-Za-z0-9+/=]+)'/)?.[1];
expect(nonce1).toBeTruthy();
expect(nonce2).toBeTruthy();
expect(nonce1).not.toBe(nonce2);
}); });
it('ten requests produce ten distinct nonces', () => { it('ten requests do not introduce nonce-bearing CSP variants', () => {
const nonces = new Set<string>(); const csps = new Set<string>();
for (let i = 0; i < 10; i++) { for (let i = 0; i < 10; i++) {
const csp = getCsp(); const csp = getCsp();
const nonce = csp.match(/'nonce-([A-Za-z0-9+/=]+)'/)?.[1]; expect(csp).not.toMatch(/'nonce-[A-Za-z0-9+/=]+'/);
expect(nonce).toBeTruthy(); csps.add(csp);
nonces.add(nonce!);
} }
expect(nonces.size).toBe(10); expect(csps.size).toBe(1);
}); });
}); });
@@ -185,8 +173,9 @@ describe('production CSP directive completeness', () => {
expect(csp).toContain("default-src 'self'"); expect(csp).toContain("default-src 'self'");
}); });
it('has script-src with nonce', () => { it('has script-src with hydration compatibility fallback', () => {
expect(csp).toMatch(/script-src [^;]*'nonce-/); expect(csp).toMatch(/script-src [^;]*'unsafe-inline'/);
expect(csp).not.toMatch(/script-src [^;]*'nonce-/);
}); });
it('has style-src with unsafe-inline and fonts.googleapis.com', () => { it('has style-src with unsafe-inline and fonts.googleapis.com', () => {
@@ -1,8 +1,9 @@
/** /**
* Phase 5F-B: Production script-src unsafe-inline removal tests. * Phase 5F-B: Production script-src nonce hardening tests.
* *
* Validates: * Validates:
* 1. Production CSP omits script-src 'unsafe-inline' * 1. Production CSP preserves hydration-safe script execution with a compatibility
* inline fallback required by the Next.js production runtime
* 2. Dev CSP retains 'unsafe-inline' and 'unsafe-eval' * 2. Dev CSP retains 'unsafe-inline' and 'unsafe-eval'
* 3. Unchanged directives (style-src, font-src, worker-src, etc.) intact * 3. Unchanged directives (style-src, font-src, worker-src, etc.) intact
* 4. API/static route exclusions remain intact * 4. API/static route exclusions remain intact
@@ -41,7 +42,7 @@ function matcherExcludes(path: string): boolean {
} }
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// 1. Production CSP omits script-src 'unsafe-inline' // 1. Production CSP stays hardened without blocking Next hydration
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
describe('production script-src hardening', () => { describe('production script-src hardening', () => {
@@ -52,9 +53,9 @@ describe('production script-src hardening', () => {
vi.unstubAllEnvs(); vi.unstubAllEnvs();
}); });
it('production script-src does NOT contain unsafe-inline', () => { it('production script-src contains unsafe-inline compatibility fallback', () => {
const scriptSrc = getDirective('script-src'); const scriptSrc = getDirective('script-src');
expect(scriptSrc).not.toContain("'unsafe-inline'"); expect(scriptSrc).toContain("'unsafe-inline'");
}); });
it('production script-src does NOT contain unsafe-eval', () => { it('production script-src does NOT contain unsafe-eval', () => {
@@ -62,9 +63,9 @@ describe('production script-src hardening', () => {
expect(scriptSrc).not.toContain("'unsafe-eval'"); expect(scriptSrc).not.toContain("'unsafe-eval'");
}); });
it('production script-src contains nonce', () => { it('production script-src does not contain nonce until all Next inline scripts are wired', () => {
const scriptSrc = getDirective('script-src'); const scriptSrc = getDirective('script-src');
expect(scriptSrc).toMatch(/'nonce-[A-Za-z0-9+/=]+'/); expect(scriptSrc).not.toMatch(/'nonce-[A-Za-z0-9+/=]+'/);
}); });
it('production script-src contains self and blob:', () => { it('production script-src contains self and blob:', () => {
@@ -105,9 +106,9 @@ describe('dev script-src allowances', () => {
expect(scriptSrc).toContain("'unsafe-eval'"); expect(scriptSrc).toContain("'unsafe-eval'");
}); });
it('dev script-src still contains nonce', () => { it('dev script-src also omits nonce to match production hydration behavior', () => {
const scriptSrc = getDirective('script-src'); const scriptSrc = getDirective('script-src');
expect(scriptSrc).toMatch(/'nonce-[A-Za-z0-9+/=]+'/); expect(scriptSrc).not.toMatch(/'nonce-[A-Za-z0-9+/=]+'/);
}); });
it('dev connect-src includes localhost backends', () => { it('dev connect-src includes localhost backends', () => {
@@ -213,10 +214,12 @@ describe('per-request environment evaluation', () => {
it('switching NODE_ENV between calls changes script-src', () => { it('switching NODE_ENV between calls changes script-src', () => {
vi.stubEnv('NODE_ENV', 'production'); vi.stubEnv('NODE_ENV', 'production');
const prodScriptSrc = getDirective('script-src'); const prodScriptSrc = getDirective('script-src');
expect(prodScriptSrc).not.toContain("'unsafe-inline'"); expect(prodScriptSrc).toContain("'unsafe-inline'");
expect(prodScriptSrc).not.toContain("'unsafe-eval'");
vi.stubEnv('NODE_ENV', 'development'); vi.stubEnv('NODE_ENV', 'development');
const devScriptSrc = getDirective('script-src'); const devScriptSrc = getDirective('script-src');
expect(devScriptSrc).toContain("'unsafe-inline'"); expect(devScriptSrc).toContain("'unsafe-inline'");
expect(devScriptSrc).toContain("'unsafe-eval'");
}); });
}); });
@@ -9,12 +9,12 @@ import {
} from '@/lib/updateRuntime'; } from '@/lib/updateRuntime';
const RELEASE: GitHubLatestRelease = { const RELEASE: GitHubLatestRelease = {
html_url: 'https://github.com/BigBodyCobain/Shadowbroker/releases/tag/v0.9.7', html_url: 'https://github.com/BigBodyCobain/Shadowbroker/releases/tag/v0.9.75',
assets: [ assets: [
{ name: 'ShadowBroker_0.9.7_x64_en-US.msi', browser_download_url: 'https://example.test/windows.msi' }, { name: 'ShadowBroker_0.9.75_x64_en-US.msi', browser_download_url: 'https://example.test/windows.msi' },
{ name: 'ShadowBroker_0.9.7_x64-setup.exe', browser_download_url: 'https://example.test/windows-setup.exe' }, { name: 'ShadowBroker_0.9.75_x64-setup.exe', browser_download_url: 'https://example.test/windows-setup.exe' },
{ name: 'ShadowBroker_0.9.7_aarch64.dmg', browser_download_url: 'https://example.test/macos.dmg' }, { name: 'ShadowBroker_0.9.75_aarch64.dmg', browser_download_url: 'https://example.test/macos.dmg' },
{ name: 'ShadowBroker_0.9.7_amd64.AppImage', browser_download_url: 'https://example.test/linux.AppImage' }, { name: 'ShadowBroker_0.9.75_amd64.AppImage', browser_download_url: 'https://example.test/linux.AppImage' },
], ],
}; };
@@ -179,8 +179,10 @@ describe('MeshChat decomposition — identity persistence', () => {
const controller = readFile('useMeshChatController.ts'); const controller = readFile('useMeshChatController.ts');
expect(controller).toMatch(/from\s+['"]@\/mesh\/meshIdentity['"]/); expect(controller).toMatch(/from\s+['"]@\/mesh\/meshIdentity['"]/);
expect(controller).toMatch(/getNodeIdentity/); expect(controller).toMatch(/getNodeIdentity/);
expect(controller).toMatch(/generateNodeKeys/); expect(controller).toMatch(/getStoredNodeDescriptor/);
expect(controller).toMatch(/signEvent/); expect(controller).toMatch(/nextSequence/);
expect(controller).toMatch(/verifyEventSignature/);
expect(controller).toMatch(/setSecureModeCached/);
}); });
it('storage module imports from meshIdentity for seal operations', () => { it('storage module imports from meshIdentity for seal operations', () => {
@@ -2,7 +2,7 @@ import '@testing-library/jest-dom/vitest';
import React from 'react'; import React from 'react';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { cleanup, fireEvent, render, screen } from '@testing-library/react'; import { cleanup, fireEvent, render, screen, waitFor } from '@testing-library/react';
let contactsState: Record<string, any> = {}; let contactsState: Record<string, any> = {};
@@ -300,12 +300,14 @@ describe('MessagesView first-contact trust UX', () => {
renderMessagesView(); renderMessagesView();
await openComposeForRecipient('!sb_pinned', 'hello after warmup'); await openComposeForRecipient('!sb_pinned', 'hello after warmup');
fireEvent.click(screen.getByRole('button', { name: 'Send Secure Mail' })); const sendButton = screen.getByRole('button', { name: 'Send Secure Mail' });
await waitFor(() => expect(sendButton).toBeEnabled(), { timeout: 5000 });
fireEvent.click(sendButton);
await screen.findByText(/Mail delivered to Pinned Peer/i); await waitFor(() => expect(mocks.prepareWormholeInteractiveLane).toHaveBeenCalled(), { timeout: 5000 });
expect(mocks.prepareWormholeInteractiveLane).toHaveBeenCalled(); await waitFor(() => expect(mocks.sendDmMessage).toHaveBeenCalled(), { timeout: 5000 });
expect(mocks.sendDmMessage).toHaveBeenCalled(); await screen.findByText(/Mail delivered to Pinned Peer/i, {}, { timeout: 5000 });
}); }, 10000);
it('does not flatten witness policy not met into a generic witnessed root label', async () => { it('does not flatten witness policy not met into a generic witnessed root label', async () => {
contactsState = { contactsState = {
+38 -10
View File
@@ -26,6 +26,8 @@ const STRIP_REQUEST = new Set([
'transfer-encoding', 'transfer-encoding',
'upgrade', 'upgrade',
'host', 'host',
'content-length',
'expect',
]); ]);
// Headers that must not be forwarded back to the browser. // Headers that must not be forwarded back to the browser.
@@ -51,6 +53,10 @@ const NO_STORE_PROXY_HEADERS = {
Pragma: 'no-cache', Pragma: 'no-cache',
}; };
function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
function isSensitiveProxyPath(pathSegments: string[]): boolean { function isSensitiveProxyPath(pathSegments: string[]): boolean {
const joined = pathSegments.join('/'); const joined = pathSegments.join('/');
if (!joined) return false; if (!joined) return false;
@@ -60,6 +66,7 @@ function isSensitiveProxyPath(pathSegments: string[]): boolean {
if (joined === 'system/update') return true; if (joined === 'system/update') return true;
if (pathSegments[0] === 'settings') return true; if (pathSegments[0] === 'settings') return true;
if (joined === 'mesh/infonet/ingest') return true; if (joined === 'mesh/infonet/ingest') return true;
if (joined === 'mesh/meshtastic/send') return true;
// mesh/peers and all tools/* use require_local_operator on the backend and // mesh/peers and all tools/* use require_local_operator on the backend and
// need X-Admin-Key injected on the server-side proxy leg. // need X-Admin-Key injected on the server-side proxy leg.
if (pathSegments[0] === 'mesh' && pathSegments[1] === 'peers') return true; if (pathSegments[0] === 'mesh' && pathSegments[1] === 'peers') return true;
@@ -76,8 +83,7 @@ async function proxy(req: NextRequest, pathSegments: string[]): Promise<NextResp
isMesh && isMesh &&
!isSensitiveMeshPath && !isSensitiveMeshPath &&
['POST', 'PUT', 'DELETE'].includes(req.method.toUpperCase()) && ['POST', 'PUT', 'DELETE'].includes(req.method.toUpperCase()) &&
(meshSegments.join('/') === 'send' || (meshSegments.join('/') === 'vote' ||
meshSegments.join('/') === 'vote' ||
meshSegments.join('/') === 'report' || meshSegments.join('/') === 'report' ||
meshSegments.join('/') === 'gate/create' || meshSegments.join('/') === 'gate/create' ||
(meshSegments[0] === 'gate' && meshSegments[2] === 'message') || (meshSegments[0] === 'gate' && meshSegments[2] === 'message') ||
@@ -191,23 +197,45 @@ async function proxy(req: NextRequest, pathSegments: string[]): Promise<NextResp
} }
const isBodyless = req.method === 'GET' || req.method === 'HEAD'; const isBodyless = req.method === 'GET' || req.method === 'HEAD';
let upstream: Response; let upstream: Response | null = null;
const requestInit: RequestInit & { duplex?: 'half' } = { const requestInit: RequestInit & { duplex?: 'half' } = {
method: req.method, method: req.method,
headers: forwardHeaders, headers: forwardHeaders,
cache: 'no-store', cache: 'no-store',
}; };
if (!isBodyless) { if (!isBodyless) {
requestInit.body = req.body; const body = await req.text();
// Required for streaming request bodies in Node.js fetch if (body.length > 0) {
requestInit.duplex = 'half'; requestInit.body = body;
}
} }
try { const maxAttempts = isBodyless ? 18 : 1;
upstream = await fetch(targetUrl.toString(), requestInit); let fetchError: unknown = null;
} catch { for (let attempt = 1; attempt <= maxAttempts; attempt += 1) {
try {
upstream = await fetch(targetUrl.toString(), requestInit);
fetchError = null;
break;
} catch (error) {
fetchError = error;
if (attempt >= maxAttempts) {
console.error('api proxy upstream fetch failed', {
method: req.method,
target: targetUrl.toString(),
error,
});
}
if (attempt >= maxAttempts) break;
await sleep(250);
}
}
if (!upstream) {
return new NextResponse(JSON.stringify({ error: 'Backend unavailable' }), { return new NextResponse(JSON.stringify({ error: 'Backend unavailable' }), {
status: 502, status: 502,
headers: { 'Content-Type': 'application/json' }, headers: {
'Content-Type': 'application/json',
'X-Proxy-Error': fetchError instanceof Error ? fetchError.name : 'fetch_failed',
},
}); });
} }
+6
View File
@@ -8,6 +8,12 @@ export const metadata: Metadata = {
description: 'Advanced Geopolitical Risk Dashboard', description: 'Advanced Geopolitical Risk Dashboard',
}; };
// The dashboard is a live local runtime, not a static landing page. If Next
// prerenders and caches the initial shell, Docker users can get stuck on the
// "prioritizing map feeds" markup before client polling ever hydrates.
export const dynamic = 'force-dynamic';
export const revalidate = 0;
export default function RootLayout({ export default function RootLayout({
children, children,
}: Readonly<{ }: Readonly<{
+140 -73
View File
@@ -26,11 +26,12 @@ import GlobalTicker from '@/components/GlobalTicker';
import ErrorBoundary from '@/components/ErrorBoundary'; import ErrorBoundary from '@/components/ErrorBoundary';
import OnboardingModal, { useOnboarding } from '@/components/OnboardingModal'; import OnboardingModal, { useOnboarding } from '@/components/OnboardingModal';
import ChangelogModal, { useChangelog } from '@/components/ChangelogModal'; import ChangelogModal, { useChangelog } from '@/components/ChangelogModal';
import StartupWarmupModal, { useStartupWarmupNotice } from '@/components/StartupWarmupModal';
import type { ActiveLayers, KiwiSDR, Scanner, SelectedEntity } from '@/types/dashboard'; import type { ActiveLayers, KiwiSDR, Scanner, SelectedEntity } from '@/types/dashboard';
import type { ShodanSearchMatch } from '@/types/shodan'; import type { ShodanSearchMatch } from '@/types/shodan';
import { API_BASE } from '@/lib/api'; import { API_BASE } from '@/lib/api';
import { useDataPolling, LAYER_TOGGLE_EVENT } from '@/hooks/useDataPolling'; import { useDataPolling, LAYER_TOGGLE_EVENT } from '@/hooks/useDataPolling';
import { useBackendStatus, useDataKey } from '@/hooks/useDataStore'; import { useBackendStatus, useDataKey, useDataKeys } from '@/hooks/useDataStore';
import { useReverseGeocode } from '@/hooks/useReverseGeocode'; import { useReverseGeocode } from '@/hooks/useReverseGeocode';
import { useRegionDossier } from '@/hooks/useRegionDossier'; import { useRegionDossier } from '@/hooks/useRegionDossier';
import { useAgentActions } from '@/hooks/useAgentActions'; import { useAgentActions } from '@/hooks/useAgentActions';
@@ -61,6 +62,9 @@ const MaplibreViewer = dynamic(() => import('@/components/MaplibreViewer'), { ss
export default function Dashboard() { export default function Dashboard() {
const viewBoundsRef = useRef<{ south: number; west: number; north: number; east: number } | null>(null); const viewBoundsRef = useRef<{ south: number; west: number; north: number; east: number } | null>(null);
// Start the critical map data request before panel/control-plane effects.
// Non-map widgets can warm up after this; first paint needs flights, ships, and intel first.
useDataPolling();
const { mouseCoords, locationLabel, handleMouseCoords } = useReverseGeocode(); const { mouseCoords, locationLabel, handleMouseCoords } = useReverseGeocode();
const [selectedEntity, setSelectedEntity] = useState<SelectedEntity | null>(null); const [selectedEntity, setSelectedEntity] = useState<SelectedEntity | null>(null);
const [trackedSdr, setTrackedSdr] = useState<KiwiSDR | null>(null); const [trackedSdr, setTrackedSdr] = useState<KiwiSDR | null>(null);
@@ -211,10 +215,35 @@ export default function Dashboard() {
const [shodanResults, setShodanResults] = useState<ShodanSearchMatch[]>([]); const [shodanResults, setShodanResults] = useState<ShodanSearchMatch[]>([]);
const [, setShodanQueryLabel] = useState(''); const [, setShodanQueryLabel] = useState('');
const [shodanStyle, setShodanStyle] = useState<import('@/types/shodan').ShodanStyleConfig>({ shape: 'circle', color: '#16a34a', size: 'md' }); const [shodanStyle, setShodanStyle] = useState<import('@/types/shodan').ShodanStyleConfig>({ shape: 'circle', color: '#16a34a', size: 'md' });
useDataPolling();
const backendStatus = useBackendStatus(); const backendStatus = useBackendStatus();
const spaceWeather = useDataKey('space_weather'); const spaceWeather = useDataKey('space_weather');
const feedHealth = useFeedHealth(); const feedHealth = useFeedHealth();
const bootSignals = useDataKeys([
'bootstrap_ready',
'commercial_flights',
'military_flights',
'tracked_flights',
'ships',
'news',
'threat_level',
] as const);
const criticalPaintReady = Boolean(
bootSignals.bootstrap_ready ||
(bootSignals.commercial_flights?.length || 0) > 0 ||
(bootSignals.military_flights?.length || 0) > 0 ||
(bootSignals.tracked_flights?.length || 0) > 0 ||
(bootSignals.ships?.length || 0) > 0 ||
(bootSignals.news?.length || 0) > 0 ||
bootSignals.threat_level,
);
const [secondaryBootReady, setSecondaryBootReady] = useState(false);
useEffect(() => {
if (secondaryBootReady) return;
const delay = criticalPaintReady ? 900 : 5500;
const id = window.setTimeout(() => setSecondaryBootReady(true), delay);
return () => window.clearTimeout(id);
}, [criticalPaintReady, secondaryBootReady]);
// Global keyboard shortcuts // Global keyboard shortcuts
useKeyboardShortcuts({ useKeyboardShortcuts({
@@ -249,6 +278,7 @@ export default function Dashboard() {
const layersTimerRef = useRef<ReturnType<typeof setTimeout> | null>(null); const layersTimerRef = useRef<ReturnType<typeof setTimeout> | null>(null);
const initialLayerSyncRef = useRef(false); const initialLayerSyncRef = useRef(false);
useEffect(() => { useEffect(() => {
if (!secondaryBootReady) return;
const syncLayers = (triggerRefetch: boolean) => const syncLayers = (triggerRefetch: boolean) =>
fetch(`${API_BASE}/api/layers`, { fetch(`${API_BASE}/api/layers`, {
method: 'POST', method: 'POST',
@@ -258,7 +288,7 @@ export default function Dashboard() {
if (triggerRefetch) { if (triggerRefetch) {
window.dispatchEvent(new Event(LAYER_TOGGLE_EVENT)); window.dispatchEvent(new Event(LAYER_TOGGLE_EVENT));
} }
}).catch((e) => console.error('Failed to update backend layers:', e)); }).catch((e) => console.warn('Backend layer sync will retry after runtime is reachable:', e));
if (layersTimerRef.current) clearTimeout(layersTimerRef.current); if (layersTimerRef.current) clearTimeout(layersTimerRef.current);
if (!initialLayerSyncRef.current) { if (!initialLayerSyncRef.current) {
@@ -272,7 +302,7 @@ export default function Dashboard() {
return () => { return () => {
if (layersTimerRef.current) clearTimeout(layersTimerRef.current); if (layersTimerRef.current) clearTimeout(layersTimerRef.current);
}; };
}, [activeLayers]); }, [activeLayers, secondaryBootReady]);
// Left panel accordion state // Left panel accordion state
const [leftDataMinimized, setLeftDataMinimized] = useState(false); const [leftDataMinimized, setLeftDataMinimized] = useState(false);
@@ -393,12 +423,28 @@ export default function Dashboard() {
}; };
const [activeFilters, setActiveFilters] = useState<Record<string, string[]>>({}); const [activeFilters, setActiveFilters] = useState<Record<string, string[]>>({});
const firstPaintActiveLayers = useMemo<ActiveLayers>(() => {
if (secondaryBootReady) return activeLayers;
return {
...activeLayers,
cctv: false,
sar: false,
gibs_imagery: false,
highres_satellite: false,
sentinel_hub: false,
viirs_nightlights: false,
psk_reporter: false,
tinygs: false,
datacenters: false,
power_plants: false,
};
}, [activeLayers, secondaryBootReady]);
// Agent fly_to handler (sar_focus_aoi etc.) — wired here now that // Agent fly_to handler (sar_focus_aoi etc.) — wired here now that
// setFlyToLocation is in scope. show_image is routed through // setFlyToLocation is in scope. show_image is routed through
// useAgentActions at the top of Dashboard. // useAgentActions at the top of Dashboard.
useAgentActions(handleMapRightClick, ({ lat, lng }) => { useAgentActions(handleMapRightClick, ({ lat, lng }) => {
setFlyToLocation({ lat, lng, ts: Date.now() }); setFlyToLocation({ lat, lng, ts: Date.now() });
}); }, secondaryBootReady);
// Eavesdrop Mode State // Eavesdrop Mode State
const [isEavesdropping] = useState(false); const [isEavesdropping] = useState(false);
@@ -407,6 +453,7 @@ export default function Dashboard() {
// Onboarding & connection status // Onboarding & connection status
const { showOnboarding, setShowOnboarding } = useOnboarding(); const { showOnboarding, setShowOnboarding } = useOnboarding();
const { showWarmupNotice, setShowWarmupNotice } = useStartupWarmupNotice();
const { showChangelog, setShowChangelog } = useChangelog(); const { showChangelog, setShowChangelog } = useChangelog();
return ( return (
@@ -415,7 +462,7 @@ export default function Dashboard() {
{/* MAPLIBRE WEBGL OVERLAY */} {/* MAPLIBRE WEBGL OVERLAY */}
<ErrorBoundary name="Map"> <ErrorBoundary name="Map">
<MaplibreViewer <MaplibreViewer
activeLayers={activeLayers} activeLayers={firstPaintActiveLayers}
activeFilters={activeFilters} activeFilters={activeFilters}
effects={memoizedEffects} effects={memoizedEffects}
onEntityClick={setSelectedEntity} onEntityClick={setSelectedEntity}
@@ -502,74 +549,87 @@ export default function Dashboard() {
> >
{/* 1. DATA LAYERS (Top) */} {/* 1. DATA LAYERS (Top) */}
<div className="contents" style={{ direction: 'ltr' }}> <div className="contents" style={{ direction: 'ltr' }}>
<ErrorBoundary name="WorldviewLeftPanel"> {secondaryBootReady ? (
<WorldviewLeftPanel <ErrorBoundary name="WorldviewLeftPanel">
activeLayers={activeLayers} <WorldviewLeftPanel
setActiveLayers={setActiveLayers} activeLayers={activeLayers}
shodanResultCount={shodanResults.length} setActiveLayers={setActiveLayers}
onSettingsClick={() => setSettingsOpen(true)} shodanResultCount={shodanResults.length}
onLegendClick={() => setLegendOpen(true)} onSettingsClick={() => setSettingsOpen(true)}
onOpenSarAoiEditor={() => setSarAoiEditorOpen(true)} onLegendClick={() => setLegendOpen(true)}
gibsDate={gibsDate} onOpenSarAoiEditor={() => setSarAoiEditorOpen(true)}
setGibsDate={setGibsDate} gibsDate={gibsDate}
gibsOpacity={gibsOpacity} setGibsDate={setGibsDate}
setGibsOpacity={setGibsOpacity} gibsOpacity={gibsOpacity}
sentinelDate={sentinelDate} setGibsOpacity={setGibsOpacity}
setSentinelDate={setSentinelDate} sentinelDate={sentinelDate}
sentinelOpacity={sentinelOpacity} setSentinelDate={setSentinelDate}
setSentinelOpacity={setSentinelOpacity} sentinelOpacity={sentinelOpacity}
sentinelPreset={sentinelPreset} setSentinelOpacity={setSentinelOpacity}
setSentinelPreset={setSentinelPreset} sentinelPreset={sentinelPreset}
onEntityClick={setSelectedEntity} setSentinelPreset={setSentinelPreset}
onFlyTo={handleFlyTo} onEntityClick={setSelectedEntity}
trackedSdr={trackedSdr} onFlyTo={handleFlyTo}
setTrackedSdr={setTrackedSdr} trackedSdr={trackedSdr}
trackedScanner={trackedScanner} setTrackedSdr={setTrackedSdr}
setTrackedScanner={setTrackedScanner} trackedScanner={trackedScanner}
isMinimized={leftDataMinimized} setTrackedScanner={setTrackedScanner}
onMinimizedChange={setLeftDataMinimized} isMinimized={leftDataMinimized}
/> onMinimizedChange={setLeftDataMinimized}
</ErrorBoundary> />
</ErrorBoundary>
) : (
<div className="bg-[#05090d]/95 border border-cyan-900/50 p-4 font-mono text-cyan-500/70">
<div className="text-[11px] tracking-[0.2em] text-cyan-400 font-bold">DATA LAYERS</div>
<div className="mt-3 text-[10px] tracking-wider">PRIORITIZING MAP FEEDS</div>
</div>
)}
</div> </div>
{/* 2. MESH CHAT (Middle) */} {/* 2. MESH CHAT (Middle) */}
<div className="contents" style={{ direction: 'ltr' }}> {secondaryBootReady && (
<MeshChat <div className="contents" style={{ direction: 'ltr' }}>
onFlyTo={handleFlyTo} <MeshChat
expanded={leftMeshExpanded} onFlyTo={handleFlyTo}
onExpandedChange={setLeftMeshExpanded} expanded={leftMeshExpanded}
onSettingsClick={() => setSettingsOpen(true)} onExpandedChange={setLeftMeshExpanded}
onTerminalToggle={openSecureTerminalLauncher} onSettingsClick={() => setSettingsOpen(true)}
launchRequest={meshChatLaunchRequest} onTerminalToggle={openSecureTerminalLauncher}
/> launchRequest={meshChatLaunchRequest}
</div> />
</div>
)}
{/* 3. SHODAN CONNECTOR (Bottom) */} {/* 3. SHODAN CONNECTOR (Bottom) */}
<div className="contents" style={{ direction: 'ltr' }}> {secondaryBootReady && (
<ShodanPanel <div className="contents" style={{ direction: 'ltr' }}>
currentResults={shodanResults} <ShodanPanel
onOpenSettings={() => setSettingsOpen(true)} currentResults={shodanResults}
settingsOpen={settingsOpen} onOpenSettings={() => setSettingsOpen(true)}
onResultsChange={(results, queryLabel) => { settingsOpen={settingsOpen}
setShodanResults(results); onResultsChange={(results, queryLabel) => {
setShodanQueryLabel(queryLabel); setShodanResults(results);
setActiveLayers((prev) => ({ ...prev, shodan_overlay: results.length > 0 })); setShodanQueryLabel(queryLabel);
}} setActiveLayers((prev) => ({ ...prev, shodan_overlay: results.length > 0 }));
onSelectEntity={setSelectedEntity} }}
onStyleChange={setShodanStyle} onSelectEntity={setSelectedEntity}
isMinimized={leftShodanMinimized} onStyleChange={setShodanStyle}
onMinimizedChange={setLeftShodanMinimized} isMinimized={leftShodanMinimized}
/> onMinimizedChange={setLeftShodanMinimized}
</div> />
</div>
)}
{/* 4. AI INTEL (Below Shodan) */} {/* 4. AI INTEL (Below Shodan) */}
<div className="contents" style={{ direction: 'ltr' }}> {secondaryBootReady && (
<AIIntelPanel <div className="contents" style={{ direction: 'ltr' }}>
onFlyTo={handleFlyTo} <AIIntelPanel
pinPlacementMode={pinPlacementMode} onFlyTo={handleFlyTo}
onPinPlacementModeChange={setPinPlacementMode} pinPlacementMode={pinPlacementMode}
/> onPinPlacementModeChange={setPinPlacementMode}
</div> />
</div>
)}
</motion.div> </motion.div>
{/* LEFT SIDEBAR TOGGLE TAB — aligns with Data Layers section */} {/* LEFT SIDEBAR TOGGLE TAB — aligns with Data Layers section */}
@@ -647,11 +707,13 @@ export default function Dashboard() {
{/* GLOBAL TICKER REPLACES MARKETS PANEL - RENDERED OUTSIDE THIS DIV */} {/* GLOBAL TICKER REPLACES MARKETS PANEL - RENDERED OUTSIDE THIS DIV */}
{/* EVENT TIMELINE */} {/* EVENT TIMELINE */}
<div className={`flex-shrink-0 ${rightFocusedPanel && rightFocusedPanel !== 'predictions' ? 'hidden' : ''}`}> {secondaryBootReady && (
<ErrorBoundary name="TimelinePanel"> <div className={`flex-shrink-0 ${rightFocusedPanel && rightFocusedPanel !== 'predictions' ? 'hidden' : ''}`}>
<TimelinePanel /> <ErrorBoundary name="TimelinePanel">
</ErrorBoundary> <TimelinePanel />
</div> </ErrorBoundary>
</div>
)}
{/* DATA FILTERS */} {/* DATA FILTERS */}
<div className={`flex-shrink-0 ${rightFocusedPanel && rightFocusedPanel !== 'filters' ? 'hidden' : ''}`}> <div className={`flex-shrink-0 ${rightFocusedPanel && rightFocusedPanel !== 'filters' ? 'hidden' : ''}`}>
@@ -870,8 +932,13 @@ export default function Dashboard() {
/> />
)} )}
{/* FIRST-RUN WARMUP NOTICE — shows once after onboarding */}
{!showOnboarding && showWarmupNotice && (
<StartupWarmupModal onClose={() => setShowWarmupNotice(false)} />
)}
{/* v0.4 CHANGELOG MODAL — shows once per version after onboarding */} {/* v0.4 CHANGELOG MODAL — shows once per version after onboarding */}
{!showOnboarding && showChangelog && ( {!showOnboarding && !showWarmupNotice && showChangelog && (
<ChangelogModal onClose={() => setShowChangelog(false)} /> <ChangelogModal onClose={() => setShowChangelog(false)} />
)} )}
+38 -2
View File
@@ -20,11 +20,23 @@ import {
Heart, Heart,
} from 'lucide-react'; } from 'lucide-react';
const CURRENT_VERSION = '0.9.7'; const CURRENT_VERSION = '0.9.75';
const STORAGE_KEY = `shadowbroker_changelog_v${CURRENT_VERSION}`; const STORAGE_KEY = `shadowbroker_changelog_v${CURRENT_VERSION}`;
const RELEASE_TITLE = 'Agentic AI Channel + InfoNet Decentralized Intelligence'; const RELEASE_TITLE = 'Onboarding, Live Feeds, Mesh, and Agent Hardening';
const HEADLINE_FEATURES = [ const HEADLINE_FEATURES = [
{
icon: <Bot size={20} className="text-purple-400" />,
accent: 'purple' as const,
title: 'Agentic onboarding for OpenClaw-compatible agents',
subtitle: 'First-time setup now includes local/direct agent connection, access-tier selection, copyable HMAC setup, and optional Tor hidden-service prep.',
details: [
'The onboarding flow can generate the local agent connection bundle through the existing HMAC API, point agents at /api/ai/tools, and let operators choose restricted read-only or full write access before connecting an agent.',
'Remote mode is labeled honestly: .onion exposes the signed HTTP agent API over Tor. Wormhole/MLS is not claimed as the current agent command transport.',
'The setup copy works for OpenClaw, Hermes, or any custom agent that implements the documented HMAC request contract.',
],
callToAction: 'OPEN FIRST-TIME SETUP -> AI AGENT',
},
{ {
icon: <Bot size={20} className="text-purple-400" />, icon: <Bot size={20} className="text-purple-400" />,
accent: 'purple' as const, accent: 'purple' as const,
@@ -53,6 +65,26 @@ const HEADLINE_FEATURES = [
]; ];
const NEW_FEATURES = [ const NEW_FEATURES = [
{
icon: <Clock size={18} className="text-cyan-400" />,
title: 'Startup and Feed Responsiveness Pass',
desc: 'Map-critical feeds now lean on startup caches and priority preload behavior so the dashboard can paint before heavyweight synthesis jobs finish.',
},
{
icon: <Network size={18} className="text-green-400" />,
title: 'MeshChat MQTT Settings',
desc: 'Public MeshChat stays opt-in and now has an in-panel settings lane for broker, port, username, password, and channel PSK while remaining separated from Wormhole/private mode.',
},
{
icon: <Plane size={18} className="text-cyan-400" />,
title: 'Selected Entity Trails',
desc: 'Flight and vessel trails are drawn only for selected assets, reducing global clutter while still exposing movement history for unknown-route entities.',
},
{
icon: <Plane size={18} className="text-amber-400" />,
title: 'Aircraft Detail Cards',
desc: 'Commercial aircraft stay airline-first, while private and general aviation aircraft can show model-focused Wiki context and imagery when available.',
},
{ {
icon: <Cpu size={18} className="text-purple-400" />, icon: <Cpu size={18} className="text-purple-400" />,
title: 'AI Batch Command Channel', title: 'AI Batch Command Channel',
@@ -101,6 +133,10 @@ const NEW_FEATURES = [
]; ];
const BUG_FIXES = [ const BUG_FIXES = [
'Docker proxy and backend port handling hardened so changing the host backend port does not require changing the internal service contract.',
'Global Threat Intercept and live-data startup paths no longer wait on slow-tier synthesis before cached data can paint the UI.',
'MeshChat and Infonet statuses now separate public MQTT participation, private Wormhole mode, and local node bootstrap so the UI does not imply the wrong connection state.',
'Commercial aircraft detail cards no longer show a confusing model image alongside the airline card.',
'Sovereign Shell adaptive polling — voting and challenge windows refresh every 8 seconds while active, every 30 to 60 seconds when idle. Voting feels live without a websocket layer.', 'Sovereign Shell adaptive polling — voting and challenge windows refresh every 8 seconds while active, every 30 to 60 seconds when idle. Voting feels live without a websocket layer.',
'Per-row write actions (petitions, upgrades, disputes) hold isolated submission state so concurrent forms no longer share a single in-flight slot.', 'Per-row write actions (petitions, upgrades, disputes) hold isolated submission state so concurrent forms no longer share a single in-flight slot.',
'Verbatim diagnostic surfacing on every write button. The backend reason text is always shown on rejection — no opaque "denied" toasts.', 'Verbatim diagnostic surfacing on every write button. The backend reason text is always shown on rejection — no opaque "denied" toasts.',
+10 -3
View File
@@ -8,8 +8,11 @@ export interface HlsVideoHandle {
get paused(): boolean; get paused(): boolean;
} }
const HlsVideo = forwardRef<HlsVideoHandle, { url: string; className?: string; onError?: () => void }>( const HlsVideo = forwardRef<
({ url, className, onError }, ref) => { HlsVideoHandle,
{ url: string; className?: string; onError?: () => void; onLoaded?: () => void }
>(
({ url, className, onError, onLoaded }, ref) => {
const videoRef = useRef<HTMLVideoElement>(null); const videoRef = useRef<HTMLVideoElement>(null);
useImperativeHandle(ref, () => ({ useImperativeHandle(ref, () => ({
@@ -35,6 +38,7 @@ const HlsVideo = forwardRef<HlsVideoHandle, { url: string; className?: string; o
hls.on(Hls.Events.ERROR, (_e: unknown, data: { fatal?: boolean }) => { hls.on(Hls.Events.ERROR, (_e: unknown, data: { fatal?: boolean }) => {
if (data.fatal) onError?.(); if (data.fatal) onError?.();
}); });
hls.on(Hls.Events.MANIFEST_PARSED, () => onLoaded?.());
hls.loadSource(url); hls.loadSource(url);
hls.attachMedia(video); hls.attachMedia(video);
hlsInstance = hls; hlsInstance = hls;
@@ -47,7 +51,7 @@ const HlsVideo = forwardRef<HlsVideoHandle, { url: string; className?: string; o
cancelled = true; cancelled = true;
hlsInstance?.destroy(); hlsInstance?.destroy();
}; };
}, [url, onError]); }, [url, onError, onLoaded]);
return ( return (
<video <video
@@ -56,6 +60,9 @@ const HlsVideo = forwardRef<HlsVideoHandle, { url: string; className?: string; o
muted muted
playsInline playsInline
onError={() => onError?.()} onError={() => onError?.()}
onCanPlay={() => onLoaded?.()}
onLoadedData={() => onLoaded?.()}
onPlaying={() => onLoaded?.()}
className={className} className={className}
/> />
); );
@@ -11,7 +11,6 @@ import {
} from '@/mesh/infonetEconomyClient'; } from '@/mesh/infonetEconomyClient';
import { generateNodeKeys, getNodeIdentity } from '@/mesh/meshIdentity'; import { generateNodeKeys, getNodeIdentity } from '@/mesh/meshIdentity';
import { import {
DEFAULT_INFONET_SEED_URL,
fetchInfonetNodeStatusSnapshot, fetchInfonetNodeStatusSnapshot,
setInfonetNodeEnabled, setInfonetNodeEnabled,
type InfonetNodeStatusSnapshot, type InfonetNodeStatusSnapshot,
@@ -57,9 +56,12 @@ export default function BootstrapView({ marketId, onBack }: BootstrapViewProps)
const nodeEnabled = Boolean(nodeStatus?.node_enabled); const nodeEnabled = Boolean(nodeStatus?.node_enabled);
const nodeMode = String(nodeStatus?.node_mode || 'participant').toUpperCase(); const nodeMode = String(nodeStatus?.node_mode || 'participant').toUpperCase();
const syncOutcome = String(nodeStatus?.sync_runtime?.last_outcome || 'idle').toLowerCase(); const syncOutcome = String(nodeStatus?.sync_runtime?.last_outcome || 'idle').toLowerCase();
const seedPeerCount = Number(nodeStatus?.bootstrap?.default_sync_peer_count || 0); const seedPeerCount = Number(
nodeStatus?.bootstrap?.bootstrap_seed_peer_count ?? nodeStatus?.bootstrap?.default_sync_peer_count ?? 0,
);
const syncPeerCount = Number(nodeStatus?.bootstrap?.sync_peer_count || 0); const syncPeerCount = Number(nodeStatus?.bootstrap?.sync_peer_count || 0);
const lastPeerUrl = String(nodeStatus?.sync_runtime?.last_peer_url || '').trim(); const lastPeerUrl = String(nodeStatus?.sync_runtime?.last_peer_url || '').trim();
const privateTransportRequired = Boolean(nodeStatus?.private_transport_required);
const toggleNode = useCallback(async (enabled: boolean) => { const toggleNode = useCallback(async (enabled: boolean) => {
setNodeToggleBusy(true); setNodeToggleBusy(true);
@@ -146,8 +148,10 @@ export default function BootstrapView({ marketId, onBack }: BootstrapViewProps)
</div> </div>
<div className="grid grid-cols-1 md:grid-cols-3 gap-2 text-xs"> <div className="grid grid-cols-1 md:grid-cols-3 gap-2 text-xs">
<div> <div>
<div className="text-gray-500">Default Seed</div> <div className="text-gray-500">Transport</div>
<div className="text-cyan-300 font-mono break-all">{DEFAULT_INFONET_SEED_URL}</div> <div className="text-cyan-300 font-mono break-all">
{privateTransportRequired ? 'ONION / RNS ONLY' : 'CLEARNET DEV OVERRIDE'}
</div>
</div> </div>
<div> <div>
<div className="text-gray-500">Local Node</div> <div className="text-gray-500">Local Node</div>
@@ -158,15 +162,15 @@ export default function BootstrapView({ marketId, onBack }: BootstrapViewProps)
<div> <div>
<div className="text-gray-500">Sync Path</div> <div className="text-gray-500">Sync Path</div>
<div className="text-white font-mono"> <div className="text-white font-mono">
{syncPeerCount} peers / {seedPeerCount} default {syncPeerCount} peers / {seedPeerCount} seeds
</div> </div>
</div> </div>
</div> </div>
<div className="mt-3 flex flex-col md:flex-row md:items-center gap-3"> <div className="mt-3 flex flex-col md:flex-row md:items-center gap-3">
<div className="flex-1 text-[11px] text-gray-500 leading-relaxed"> <div className="flex-1 text-[11px] text-gray-500 leading-relaxed">
{nodeEnabled {nodeEnabled
? `Public chain sync is ${syncOutcome || 'active'}${lastPeerUrl ? ` via ${lastPeerUrl}` : ''}.` ? `Infonet sync is ${syncOutcome || 'active'}${lastPeerUrl ? ` via ${lastPeerUrl}` : ''}.`
: 'Start a local participant node to pull from the default seed and help carry the public Infonet chain while this backend is running.'} : 'Start a local participant node to sync through available Wormhole onion/RNS peers while this backend is running.'}
</div> </div>
<button <button
type="button" type="button"
@@ -1,7 +1,7 @@
'use client'; 'use client';
import React, { useState, useEffect, useRef, useMemo } from 'react'; import React, { useState, useEffect, useRef, useMemo } from 'react';
import { Terminal, Radio, Globe, Key, LogOut, Activity, Vote, User, ArrowRightLeft, Briefcase, Mail, Brain, GitBranch, Cpu, KeyRound } from 'lucide-react'; import { Terminal, Radio, Globe, Key, Activity, Vote, User, ArrowRightLeft, Briefcase, Mail, Brain, GitBranch, Cpu, KeyRound } from 'lucide-react';
import { getNodeIdentity, getWormholeIdentityDescriptor } from '@/mesh/meshIdentity'; import { getNodeIdentity, getWormholeIdentityDescriptor } from '@/mesh/meshIdentity';
import { import {
activateWormholeGatePersona, activateWormholeGatePersona,
@@ -128,7 +128,6 @@ const SECTIONS = [
{ name: 'EXCHANGE', icon: <ArrowRightLeft size={14} className="mr-2" /> }, { name: 'EXCHANGE', icon: <ArrowRightLeft size={14} className="mr-2" /> },
{ name: 'PROFILE', icon: <User size={14} className="mr-2" /> }, { name: 'PROFILE', icon: <User size={14} className="mr-2" /> },
{ name: 'MESSAGES', icon: <Mail size={14} className="mr-2" /> }, { name: 'MESSAGES', icon: <Mail size={14} className="mr-2" /> },
{ name: 'EXIT', icon: <LogOut size={14} className="mr-2" /> },
]; ];
interface CommandHistory { interface CommandHistory {
@@ -32,12 +32,22 @@ export default function NetworkStats() {
fetchInfonetNodeStatusSnapshot(true).catch(() => null), fetchInfonetNodeStatusSnapshot(true).catch(() => null),
]); ]);
if (!alive) return; if (!alive) return;
const knownNodes = Number(infonet?.known_nodes || 0);
const syncPeerCount = Number(infonet?.bootstrap?.sync_peer_count || 0);
const defaultSyncPeerCount = Number(infonet?.bootstrap?.default_sync_peer_count || 0);
const lastPeerUrl = String(infonet?.sync_runtime?.last_peer_url || '').trim();
const visibleInfonetNodes = Math.max(
knownNodes,
syncPeerCount,
defaultSyncPeerCount,
lastPeerUrl ? 1 : 0,
);
setStats({ setStats({
meshtastic: Number(channelsRes?.total_live || channelsRes?.total_nodes || meshRes?.signal_counts?.meshtastic || 0), meshtastic: Number(channelsRes?.total_live || channelsRes?.total_nodes || meshRes?.signal_counts?.meshtastic || 0),
aprs: Number(meshRes?.signal_counts?.aprs || 0), aprs: Number(meshRes?.signal_counts?.aprs || 0),
infonetNodes: Number(infonet?.known_nodes || 0), infonetNodes: visibleInfonetNodes,
infonetEvents: Number(infonet?.total_events || 0), infonetEvents: Number(infonet?.total_events || 0),
syncPeers: Number(infonet?.bootstrap?.sync_peer_count || 0), syncPeers: syncPeerCount,
nodeEnabled: Boolean(infonet?.node_enabled), nodeEnabled: Boolean(infonet?.node_enabled),
syncOutcome: String(infonet?.sync_runtime?.last_outcome || 'offline').toLowerCase(), syncOutcome: String(infonet?.sync_runtime?.last_outcome || 'offline').toLowerCase(),
}); });
@@ -51,7 +61,7 @@ export default function NetworkStats() {
const nodeColor = stats.syncOutcome === 'ok' ? 'text-green-400' const nodeColor = stats.syncOutcome === 'ok' ? 'text-green-400'
: stats.syncOutcome === 'running' ? 'text-amber-400' : stats.syncOutcome === 'running' ? 'text-amber-400'
: stats.nodeEnabled ? 'text-amber-400' : 'text-gray-600'; : stats.nodeEnabled ? 'text-amber-400' : 'text-gray-600';
const nodeLabel = stats.syncOutcome === 'ok' ? 'CONNECTED' const nodeLabel = stats.syncOutcome === 'ok' ? 'SEED SYNCED'
: stats.syncOutcome === 'running' ? 'SYNCING' : stats.syncOutcome === 'running' ? 'SYNCING'
: stats.syncOutcome === 'error' || stats.syncOutcome === 'fork' ? 'RETRYING' : stats.syncOutcome === 'error' || stats.syncOutcome === 'fork' ? 'RETRYING'
: stats.nodeEnabled ? 'WAITING' : 'OFFLINE'; : stats.nodeEnabled ? 'WAITING' : 'OFFLINE';
@@ -2,7 +2,12 @@
import React, { useEffect } from 'react'; import React, { useEffect } from 'react';
import { AnimatePresence, motion } from 'framer-motion'; import { AnimatePresence, motion } from 'framer-motion';
import { X, Minus } from 'lucide-react'; import { X } from 'lucide-react';
import {
fetchInfonetNodeStatusSnapshot,
setInfonetNodeEnabled,
startTorHiddenService,
} from '@/mesh/controlPlaneStatusClient';
import InfonetShell from './InfonetShell'; import InfonetShell from './InfonetShell';
interface InfonetTerminalProps { interface InfonetTerminalProps {
@@ -28,6 +33,33 @@ export default function InfonetTerminal({
return () => window.removeEventListener('keydown', handler); return () => window.removeEventListener('keydown', handler);
}, [isOpen, onClose]); }, [isOpen, onClose]);
useEffect(() => {
if (!isOpen) return;
let cancelled = false;
const connectParticipantNode = async () => {
try {
const nodeStatus = await fetchInfonetNodeStatusSnapshot(true).catch(() => null);
if (cancelled || nodeStatus?.node_enabled) return;
const torStatus = await startTorHiddenService().catch(() => null);
if (cancelled || !torStatus?.running || !torStatus?.onion_address) return;
await setInfonetNodeEnabled(true);
if (!cancelled) {
await fetchInfonetNodeStatusSnapshot(true).catch(() => null);
}
} catch {
// Remote/shared viewers may not have local-operator rights. Leave manual controls intact.
}
};
void connectParticipantNode();
return () => {
cancelled = true;
};
}, [isOpen]);
return ( return (
<AnimatePresence> <AnimatePresence>
{isOpen && ( {isOpen && (
@@ -55,13 +87,6 @@ export default function InfonetTerminal({
</span> </span>
</div> </div>
<div className="flex items-center gap-1"> <div className="flex items-center gap-1">
<button
onClick={onClose}
className="p-1 text-gray-600 hover:text-gray-300 transition-colors"
title="Minimize"
>
<Minus size={14} />
</button>
<button <button
onClick={onClose} onClick={onClose}
className="p-1 text-gray-600 hover:text-red-400 transition-colors" className="p-1 text-gray-600 hover:text-red-400 transition-colors"
+225 -33
View File
@@ -16,7 +16,7 @@ import 'maplibre-gl/dist/maplibre-gl.css';
import { computeNightPolygon } from '@/utils/solarTerminator'; import { computeNightPolygon } from '@/utils/solarTerminator';
import { darkStyle, lightStyle } from '@/components/map/styles/mapStyles'; import { darkStyle, lightStyle } from '@/components/map/styles/mapStyles';
import maplibregl from 'maplibre-gl'; import maplibregl from 'maplibre-gl';
import { AlertTriangle, Radio, Activity, Play, Satellite } from 'lucide-react'; import { AlertTriangle, Radio, Activity, Play, Satellite, ExternalLink, Info } from 'lucide-react';
import WikiImage from '@/components/WikiImage'; import WikiImage from '@/components/WikiImage';
import FishingDestinationRoute from '@/components/map/FishingDestinationRoute'; import FishingDestinationRoute from '@/components/map/FishingDestinationRoute';
import { useTheme } from '@/lib/ThemeContext'; import { useTheme } from '@/lib/ThemeContext';
@@ -159,7 +159,7 @@ import {
EarthquakeLabels, EarthquakeLabels,
ThreatMarkers, ThreatMarkers,
} from '@/components/map/MapMarkers'; } from '@/components/map/MapMarkers';
import type { DashboardData, KiwiSDR, MaplibreViewerProps, Scanner, SigintSignal } from '@/types/dashboard'; import type { DashboardData, Flight, KiwiSDR, MaplibreViewerProps, Scanner, Ship, SigintSignal } from '@/types/dashboard';
import { useDataKeys } from '@/hooks/useDataStore'; import { useDataKeys } from '@/hooks/useDataStore';
import { useInterpolation } from '@/components/map/hooks/useInterpolation'; import { useInterpolation } from '@/components/map/hooks/useInterpolation';
import { useClusterLabels } from '@/components/map/hooks/useClusterLabels'; import { useClusterLabels } from '@/components/map/hooks/useClusterLabels';
@@ -225,6 +225,63 @@ type GeoExtras = {
type KiwiProps = Partial<KiwiSDR> & GeoExtras; type KiwiProps = Partial<KiwiSDR> & GeoExtras;
type ScannerProps = Partial<Scanner> & GeoExtras; type ScannerProps = Partial<Scanner> & GeoExtras;
type SigintProps = Partial<SigintSignal> & GeoExtras; type SigintProps = Partial<SigintSignal> & GeoExtras;
type TrailPoint = { lng: number; lat: number; alt?: number; sog?: number; ts?: number };
type TrailKind = 'flight' | 'ship';
const FLIGHT_SELECTION_TYPES = new Set([
'flight',
'private_flight',
'military_flight',
'private_jet',
'tracked_flight',
]);
function parseTrailPoints(raw: unknown, kind: TrailKind): TrailPoint[] {
if (!Array.isArray(raw)) return [];
return raw
.map((p): TrailPoint | null => {
if (Array.isArray(p)) {
const lat = Number(p[0]);
const lng = Number(p[1]);
if (!Number.isFinite(lat) || !Number.isFinite(lng)) return null;
if (kind === 'ship') {
return { lat, lng, sog: Number(p[2]) || 0, ts: Number(p[3]) || 0 };
}
return { lat, lng, alt: Number(p[2]) || 0, ts: Number(p[3]) || 0 };
}
if (p && typeof p === 'object') {
const point = p as { lat?: number; lng?: number; alt?: number; sog?: number; ts?: number };
const lat = Number(point.lat);
const lng = Number(point.lng);
if (!Number.isFinite(lat) || !Number.isFinite(lng)) return null;
return {
lat,
lng,
alt: Number(point.alt) || 0,
sog: Number(point.sog) || 0,
ts: Number(point.ts) || 0,
};
}
return null;
})
.filter((p): p is TrailPoint => Boolean(p && (p.lat !== 0 || p.lng !== 0)));
}
function hasKnownRouteName(value?: string | null): boolean {
const normalized = String(value || '').trim().toUpperCase();
return Boolean(normalized && normalized !== 'UNKNOWN');
}
function flightHasKnownRoute(entity: ReturnType<typeof findSelectedEntity>, dynamicRoute: DynamicRoute | null): boolean {
if (!entity) return false;
if (dynamicRoute?.orig_loc && dynamicRoute?.dest_loc) return true;
if (!('origin_loc' in entity) && !('origin_name' in entity)) return false;
const flight = entity as Flight;
return Boolean(
(flight.origin_loc && flight.dest_loc)
|| (hasKnownRouteName(flight.origin_name) && hasKnownRouteName(flight.dest_name)),
);
}
const MAP_EXTRA_DATA_KEYS = [ const MAP_EXTRA_DATA_KEYS = [
'air_quality', 'air_quality',
@@ -293,6 +350,15 @@ function probeRasterTile(url: string): Promise<boolean> {
}); });
} }
function buildPolymarketUrl(prediction: { slug?: string; title?: string } | null | undefined): string {
const slug = String(prediction?.slug || '').trim();
if (slug) return `https://polymarket.com/event/${encodeURIComponent(slug)}`;
const title = String(prediction?.title || '').trim();
return title
? `https://polymarket.com/search?query=${encodeURIComponent(title)}`
: 'https://polymarket.com/markets';
}
const MaplibreViewer = ({ const MaplibreViewer = ({
activeLayers, activeLayers,
activeFilters, activeFilters,
@@ -336,6 +402,7 @@ const MaplibreViewer = ({
const data = useMemo(() => ({ ...coreData, ...extraData }) as DashboardData, [coreData, extraData]); const data = useMemo(() => ({ ...coreData, ...extraData }) as DashboardData, [coreData, extraData]);
const mapRef = useRef<MapRef>(null); const mapRef = useRef<MapRef>(null);
const mapInitRef = useRef(false); const mapInitRef = useRef(false);
const [mapReady, setMapReady] = useState(false);
const { theme } = useTheme(); const { theme } = useTheme();
const mapThemeStyle = useMemo<maplibregl.StyleSpecification>( const mapThemeStyle = useMemo<maplibregl.StyleSpecification>(
() => (theme === 'light' ? lightStyle : darkStyle) as maplibregl.StyleSpecification, () => (theme === 'light' ? lightStyle : darkStyle) as maplibregl.StyleSpecification,
@@ -469,6 +536,7 @@ const MaplibreViewer = ({
}, [activeLayers.viirs_nightlights, viirsProbeDayKey]); }, [activeLayers.viirs_nightlights, viirsProbeDayKey]);
const [dynamicRoute, setDynamicRoute] = useState<DynamicRoute | null>(null); const [dynamicRoute, setDynamicRoute] = useState<DynamicRoute | null>(null);
const [selectedTrailPoints, setSelectedTrailPoints] = useState<TrailPoint[]>([]);
const prevCallsign = useRef<string | null>(null); const prevCallsign = useRef<string | null>(null);
// Oracle region intel for map entity popups // Oracle region intel for map entity popups
@@ -547,6 +615,7 @@ const MaplibreViewer = ({
if (callsign && callsign !== prevCallsign.current) { if (callsign && callsign !== prevCallsign.current) {
prevCallsign.current = callsign; prevCallsign.current = callsign;
setDynamicRoute(null);
fetch(`${API_BASE}/api/route/${callsign}?lat=${entityLat}&lng=${entityLng}`) fetch(`${API_BASE}/api/route/${callsign}?lat=${entityLat}&lng=${entityLng}`)
.then((res) => res.json()) .then((res) => res.json())
.then((routeData) => { .then((routeData) => {
@@ -565,6 +634,76 @@ const MaplibreViewer = ({
}; };
}, [selectedEntity, data]); }, [selectedEntity, data]);
useEffect(() => {
let cancelled = false;
const entity = findSelectedEntity(selectedEntity, data);
if (!selectedEntity || !entity) {
setSelectedTrailPoints([]);
return () => {
cancelled = true;
};
}
const isFlight = FLIGHT_SELECTION_TYPES.has(selectedEntity.type);
const isShip = selectedEntity.type === 'ship';
if (!isFlight && !isShip) {
setSelectedTrailPoints([]);
return () => {
cancelled = true;
};
}
if (isFlight && flightHasKnownRoute(entity, dynamicRoute)) {
setSelectedTrailPoints([]);
return () => {
cancelled = true;
};
}
const kind: TrailKind = isShip ? 'ship' : 'flight';
const fallback = parseTrailPoints((entity as Flight | Ship).trail, kind);
if (fallback.length >= 2) {
setSelectedTrailPoints(fallback);
} else {
setSelectedTrailPoints([]);
}
const trailId = String(selectedEntity.id || '').trim();
if (!trailId) {
return () => {
cancelled = true;
};
}
if (isShip && !/^\d+$/.test(trailId)) {
return () => {
cancelled = true;
};
}
const endpoint = isShip
? `${API_BASE}/api/trail/ship/${encodeURIComponent(trailId)}`
: `${API_BASE}/api/trail/flight/${encodeURIComponent(trailId)}`;
const refreshSelectedTrail = () => {
fetch(endpoint, { cache: 'no-store' })
.then((res) => (res.ok ? res.json() : null))
.then((payload) => {
if (cancelled || !payload) return;
const points = parseTrailPoints(payload.trail, kind);
setSelectedTrailPoints(points.length >= 2 ? points : fallback);
})
.catch(() => {
if (!cancelled) setSelectedTrailPoints(fallback);
});
};
refreshSelectedTrail();
const trailRefreshTimer = window.setInterval(refreshSelectedTrail, 30000);
return () => {
cancelled = true;
window.clearInterval(trailRefreshTimer);
};
}, [selectedEntity, data, dynamicRoute]);
// Fetch oracle region intel for entity popups // Fetch oracle region intel for entity popups
useEffect(() => { useEffect(() => {
if (!selectedEntity) { if (!selectedEntity) {
@@ -905,15 +1044,20 @@ const MaplibreViewer = ({
// Load Images into the Map Style once loaded // Load Images into the Map Style once loaded
const onMapLoad = useCallback((e: { target: maplibregl.Map }) => { const onMapLoad = useCallback((e: { target: maplibregl.Map }) => {
initializeMap(e.target); initializeMap(e.target);
setMapReady(true);
}, [initializeMap]); }, [initializeMap]);
const onMapStyleData = useCallback((e: { target: maplibregl.Map }) => { const onMapStyleData = useCallback((e: { target: maplibregl.Map }) => {
initializeMap(e.target); initializeMap(e.target);
setMapReady(true);
}, [initializeMap]); }, [initializeMap]);
useEffect(() => { useEffect(() => {
const map = mapRef.current?.getMap(); const map = mapRef.current?.getMap();
if (map) initializeMap(map); if (map) {
initializeMap(map);
setMapReady(true);
}
}, [initializeMap, theme]); }, [initializeMap, theme]);
// Build a set of tracked icao24s to exclude from other flight layers // Build a set of tracked icao24s to exclude from other flight layers
@@ -1334,27 +1478,29 @@ const MaplibreViewer = ({
return { type: 'FeatureCollection' as const, features }; return { type: 'FeatureCollection' as const, features };
}, [selectedEntity, data, dynamicRoute, getSelectedEntityLiveCoords, interpTick]); }, [selectedEntity, data, dynamicRoute, getSelectedEntityLiveCoords, interpTick]);
// Trail history GeoJSON: shows where the SELECTED aircraft has been // Trail history GeoJSON: shows where the selected unknown-route aircraft or vessel has been.
const trailGeoJSON = useMemo(() => { const trailGeoJSON = useMemo(() => {
void interpTick; void interpTick;
const entity = findSelectedEntity(selectedEntity, data); const entity = findSelectedEntity(selectedEntity, data);
if (!entity || !('trail' in entity) || !entity.trail || entity.trail.length < 2) return null; if (!entity || selectedTrailPoints.length < 2) return null;
if (selectedEntity && FLIGHT_SELECTION_TYPES.has(selectedEntity.type) && flightHasKnownRoute(entity, dynamicRoute)) {
return null;
}
// Parse trail points — backend sends [lat, lng, alt, ts] arrays // Trails are loaded only for the selected asset to avoid open-map clutter.
type TrailPt = { lng: number; lat: number; alt: number; ts: number }; const isShipTrail = selectedEntity?.type === 'ship';
const points: TrailPt[] = ( const points = [...selectedTrailPoints];
entity.trail as Array<{ lat?: number; lng?: number; alt?: number; ts?: number } | number[]>
).map((p) => {
if (Array.isArray(p)) {
return { lat: p[0] as number, lng: p[1] as number, alt: (p[2] as number) || 0, ts: (p[3] as number) || 0 };
}
return { lat: p.lat ?? 0, lng: p.lng ?? 0, alt: p.alt ?? 0, ts: p.ts ?? 0 };
}).filter((p) => p.lat !== 0 || p.lng !== 0);
const currentLoc = getSelectedEntityLiveCoords(entity); const currentLoc = getSelectedEntityLiveCoords(entity);
if (currentLoc && points.length > 0) { if (currentLoc && points.length > 0) {
const lastPt = points[points.length - 1]; const lastPt = points[points.length - 1];
points.push({ lng: currentLoc[0], lat: currentLoc[1], alt: lastPt.alt, ts: Date.now() / 1000 }); points.push({
lng: currentLoc[0],
lat: currentLoc[1],
alt: lastPt.alt,
sog: lastPt.sog,
ts: Date.now() / 1000,
});
} }
if (points.length < 2) return null; if (points.length < 2) return null;
@@ -1379,7 +1525,7 @@ const MaplibreViewer = ({
type: 'Feature' as const, type: 'Feature' as const,
properties: { properties: {
type: 'trail', type: 'trail',
color: altToColor((a.alt + b.alt) / 2), color: isShipTrail ? '#22d3ee' : altToColor(((a.alt ?? 0) + (b.alt ?? 0)) / 2),
opacity: 0.4 + progress * 0.5, // older segments more transparent opacity: 0.4 + progress * 0.5, // older segments more transparent
segIndex: i, segIndex: i,
}, },
@@ -1391,7 +1537,7 @@ const MaplibreViewer = ({
} }
return { type: 'FeatureCollection' as const, features }; return { type: 'FeatureCollection' as const, features };
}, [selectedEntity, data, getSelectedEntityLiveCoords, interpTick]); }, [selectedEntity, data, selectedTrailPoints, dynamicRoute, getSelectedEntityLiveCoords, interpTick]);
// Predictive vector GeoJSON: dotted line projecting ~5 min ahead based on heading + speed // Predictive vector GeoJSON: dotted line projecting ~5 min ahead based on heading + speed
// Skip when entity has a known route (origin+dest) — the route line already shows where it's going // Skip when entity has a known route (origin+dest) — the route line already shows where it's going
@@ -1552,7 +1698,7 @@ const MaplibreViewer = ({
}, [activeLayers.uap_sightings, activeLayers.wastewater, theme]); }, [activeLayers.uap_sightings, activeLayers.wastewater, theme]);
// --- Imperative source updates: bypass React reconciliation for GeoJSON layers --- // --- Imperative source updates: bypass React reconciliation for GeoJSON layers ---
const mapForHook = mapRef.current; const mapForHook = mapReady ? mapRef.current : null;
useImperativeSource(mapForHook, 'commercial-flights', commFlightsGeoJSON); useImperativeSource(mapForHook, 'commercial-flights', commFlightsGeoJSON);
useImperativeSource(mapForHook, 'private-flights', privFlightsGeoJSON); useImperativeSource(mapForHook, 'private-flights', privFlightsGeoJSON);
useImperativeSource(mapForHook, 'private-jets', privJetsGeoJSON); useImperativeSource(mapForHook, 'private-jets', privJetsGeoJSON);
@@ -5712,29 +5858,56 @@ const MaplibreViewer = ({
<div className="px-5 pb-3"> <div className="px-5 pb-3">
<div className="grid grid-cols-3 gap-2"> <div className="grid grid-cols-3 gap-2">
{/* Oracle Score */} {/* Oracle Score */}
<div className={`border rounded p-3 text-center ${oTierBg || 'bg-black/40 border-cyan-800/30'}`}> <label className={`border rounded p-3 text-center transition-colors hover:border-white/40 cursor-pointer ${oTierBg || 'bg-black/40 border-cyan-800/30'}`}>
<div className="text-[9px] text-[var(--text-muted)] tracking-[0.15em] mb-1.5">ORACLE SCORE</div> <input type="checkbox" className="peer sr-only" aria-label="Explain Oracle Score" />
<div className="flex items-center justify-center gap-1 text-[9px] text-[var(--text-muted)] tracking-[0.15em] mb-1.5">
<span>ORACLE SCORE</span>
<Info size={10} />
</div>
<div className={`text-[28px] font-bold leading-none ${oTierColor || 'text-gray-500'}`}> <div className={`text-[28px] font-bold leading-none ${oTierColor || 'text-gray-500'}`}>
{oScore != null ? oScore.toFixed(1) : '—'} {oScore != null ? oScore.toFixed(1) : '—'}
</div> </div>
{oTier && <div className={`text-[10px] font-bold ${oTierColor} mt-1`}>{oTier}</div>} {oTier && <div className={`text-[10px] font-bold ${oTierColor} mt-1`}>{oTier}</div>}
</div> <div className="hidden peer-checked:block mt-2 border-t border-white/10 pt-2 text-left text-[10px] leading-relaxed text-cyan-100">
<div className="text-cyan-400 font-bold tracking-[0.16em] mb-1">SCALE</div>
<p>0-10 weighted signal score combining alert risk and source confidence.</p>
<p className="mt-1 text-[var(--text-muted)]">0-3 low, 4-5 moderate, 6-7 elevated, 8-10 critical.</p>
</div>
</label>
{/* Sentiment */} {/* Sentiment */}
<div className={`border rounded p-3 text-center ${sentBg || 'bg-black/40 border-cyan-800/30'}`}> <label className={`border rounded p-3 text-center transition-colors hover:border-white/40 cursor-pointer ${sentBg || 'bg-black/40 border-cyan-800/30'}`}>
<div className="text-[9px] text-[var(--text-muted)] tracking-[0.15em] mb-1.5">SENTIMENT</div> <input type="checkbox" className="peer sr-only" aria-label="Explain Sentiment" />
<div className="flex items-center justify-center gap-1 text-[9px] text-[var(--text-muted)] tracking-[0.15em] mb-1.5">
<span>SENTIMENT</span>
<Info size={10} />
</div>
<div className={`text-[28px] font-bold leading-none ${sentColor || 'text-gray-500'}`}> <div className={`text-[28px] font-bold leading-none ${sentColor || 'text-gray-500'}`}>
{sent != null ? <>{sentArrow} {sent > 0 ? '+' : ''}{sent.toFixed(2)}</> : '—'} {sent != null ? <>{sentArrow} {sent > 0 ? '+' : ''}{sent.toFixed(2)}</> : '—'}
</div> </div>
{sentLabel && <div className={`text-[10px] font-bold ${sentColor} mt-1`}>{sentLabel}</div>} {sentLabel && <div className={`text-[10px] font-bold ${sentColor} mt-1`}>{sentLabel}</div>}
</div> <div className="hidden peer-checked:block mt-2 border-t border-white/10 pt-2 text-left text-[10px] leading-relaxed text-cyan-100">
<div className="text-cyan-400 font-bold tracking-[0.16em] mb-1">SCALE</div>
<p>-1.00 to +1.00 headline tone. Negative reads more adverse; positive reads more constructive.</p>
<p className="mt-1 text-[var(--text-muted)]">Below -0.10 negative, -0.10 to +0.10 neutral, above +0.10 positive. It measures tone, not truth.</p>
</div>
</label>
{/* Threat Level */} {/* Threat Level */}
<div className={`border rounded p-3 text-center ${rs >= 8 ? 'bg-red-500/10 border-red-500/30' : rs >= 6 ? 'bg-orange-500/10 border-orange-500/30' : rs >= 4 ? 'bg-yellow-500/10 border-yellow-500/30' : 'bg-green-500/10 border-green-500/30'}`}> <label className={`border rounded p-3 text-center transition-colors hover:border-white/40 cursor-pointer ${rs >= 8 ? 'bg-red-500/10 border-red-500/30' : rs >= 6 ? 'bg-orange-500/10 border-orange-500/30' : rs >= 4 ? 'bg-yellow-500/10 border-yellow-500/30' : 'bg-green-500/10 border-green-500/30'}`}>
<div className="text-[9px] text-[var(--text-muted)] tracking-[0.15em] mb-1.5">RISK LEVEL</div> <input type="checkbox" className="peer sr-only" aria-label="Explain Risk Level" />
<div className="flex items-center justify-center gap-1 text-[9px] text-[var(--text-muted)] tracking-[0.15em] mb-1.5">
<span>RISK LEVEL</span>
<Info size={10} />
</div>
<div className={`text-[28px] font-bold leading-none ${threatColor}`}>{rs}/10</div> <div className={`text-[28px] font-bold leading-none ${threatColor}`}>{rs}/10</div>
<div className={`text-[10px] font-bold ${threatColor} mt-1`}> <div className={`text-[10px] font-bold ${threatColor} mt-1`}>
{rs >= 9 ? 'CRITICAL' : rs >= 7 ? 'HIGH' : rs >= 4 ? 'MEDIUM' : 'LOW'} {rs >= 9 ? 'CRITICAL' : rs >= 7 ? 'HIGH' : rs >= 4 ? 'MEDIUM' : 'LOW'}
</div> </div>
</div> <div className="hidden peer-checked:block mt-2 border-t border-white/10 pt-2 text-left text-[10px] leading-relaxed text-cyan-100">
<div className="text-cyan-400 font-bold tracking-[0.16em] mb-1">SCALE</div>
<p>0-10 operational severity estimate based on source, topic, keywords, corroboration, and alert context.</p>
<p className="mt-1 text-[var(--text-muted)]">0-3 low, 4-6 medium, 7-8 high, 9-10 critical.</p>
</div>
</label>
</div> </div>
</div> </div>
@@ -5742,8 +5915,20 @@ const MaplibreViewer = ({
{pred && pred.consensus_pct != null && ( {pred && pred.consensus_pct != null && (
<div className="px-5 pb-3"> <div className="px-5 pb-3">
<div className="bg-purple-950/30 border border-purple-500/40 rounded p-4"> <div className="bg-purple-950/30 border border-purple-500/40 rounded p-4">
<div className="text-[10px] text-purple-400 tracking-[0.2em] font-bold mb-2"> <div className="flex items-center justify-between gap-3 mb-2">
PREDICTION MARKET ANALYSIS <div className="text-[10px] text-purple-400 tracking-[0.2em] font-bold">
PREDICTION MARKET ANALYSIS
</div>
{pred.polymarket_pct != null && (
<button
type="button"
onClick={() => window.open(buildPolymarketUrl(pred), '_blank', 'noopener,noreferrer')}
className="inline-flex items-center gap-1 text-[10px] text-purple-200 hover:text-white border border-purple-500/30 hover:border-purple-300/70 px-2 py-1 rounded transition-colors"
title="Open this market on Polymarket"
>
POLYMARKET <ExternalLink size={10} />
</button>
)}
</div> </div>
<div className="text-[14px] text-purple-200 font-bold leading-snug mb-3"> <div className="text-[14px] text-purple-200 font-bold leading-snug mb-3">
&quot;{pred.title}&quot; &quot;{pred.title}&quot;
@@ -5760,10 +5945,16 @@ const MaplibreViewer = ({
</div> </div>
<div className="flex gap-6 text-[11px]"> <div className="flex gap-6 text-[11px]">
{pred.polymarket_pct != null && ( {pred.polymarket_pct != null && (
<div className="flex items-center gap-2"> <button
type="button"
onClick={() => window.open(buildPolymarketUrl(pred), '_blank', 'noopener,noreferrer')}
className="flex items-center gap-2 hover:text-white transition-colors"
title="Open this market on Polymarket"
>
<span className="text-purple-400/70">Polymarket</span> <span className="text-purple-400/70">Polymarket</span>
<span className="text-white font-bold text-[13px]">{pred.polymarket_pct}%</span> <span className="text-white font-bold text-[13px]">{pred.polymarket_pct}%</span>
</div> <ExternalLink size={10} className="text-purple-400/70" />
</button>
)} )}
{pred.kalshi_pct != null && ( {pred.kalshi_pct != null && (
<div className="flex items-center gap-2"> <div className="flex items-center gap-2">
@@ -5834,7 +6025,7 @@ const MaplibreViewer = ({
onClick={() => window.open(item.link, '_blank', 'noopener,noreferrer')} onClick={() => window.open(item.link, '_blank', 'noopener,noreferrer')}
className={`${threatColor} hover:text-white text-[12px] font-bold underline underline-offset-2 cursor-pointer`} className={`${threatColor} hover:text-white text-[12px] font-bold underline underline-offset-2 cursor-pointer`}
> >
VIEW FULL REPORT GO TO ARTICLE
</button> </button>
) : <span />} ) : <span />}
<button <button
@@ -5902,6 +6093,7 @@ const MaplibreViewer = ({
return ( return (
<CctvFullscreenModal <CctvFullscreenModal
url={url} url={url}
rawUrl={rawUrl}
mediaType={mt} mediaType={mt}
isVideo={isVideo} isVideo={isVideo}
cameraName={cameraName} cameraName={cameraName}
@@ -1,11 +1,12 @@
'use client'; 'use client';
import React, { useState, useCallback, useRef } from 'react'; import React, { useState, useCallback, useRef, useEffect, useMemo } from 'react';
import { AlertTriangle, Play, Pause } from 'lucide-react'; import { AlertTriangle, Play, Pause } from 'lucide-react';
import HlsVideo, { type HlsVideoHandle } from '@/components/HlsVideo'; import HlsVideo, { type HlsVideoHandle } from '@/components/HlsVideo';
export interface CctvFullscreenModalProps { export interface CctvFullscreenModalProps {
url: string; url: string;
rawUrl?: string;
mediaType: string; mediaType: string;
isVideo: boolean; isVideo: boolean;
cameraName: string; cameraName: string;
@@ -16,6 +17,7 @@ export interface CctvFullscreenModalProps {
export function CctvFullscreenModal({ export function CctvFullscreenModal({
url, url,
rawUrl = '',
mediaType, mediaType,
isVideo, isVideo,
cameraName, cameraName,
@@ -25,8 +27,60 @@ export function CctvFullscreenModal({
}: CctvFullscreenModalProps) { }: CctvFullscreenModalProps) {
const [paused, setPaused] = useState(false); const [paused, setPaused] = useState(false);
const [mediaError, setMediaError] = useState(false); const [mediaError, setMediaError] = useState(false);
const [mediaLoaded, setMediaLoaded] = useState(false);
const [sourceIndex, setSourceIndex] = useState(0);
const videoRef = useRef<HTMLVideoElement>(null); const videoRef = useRef<HTMLVideoElement>(null);
const hlsRef = useRef<HlsVideoHandle>(null); const hlsRef = useRef<HlsVideoHandle>(null);
const sources = useMemo(() => {
const seen = new Set<string>();
return [url, rawUrl]
.map((candidate) => String(candidate || '').trim())
.filter((candidate) => {
if (!candidate || seen.has(candidate)) return false;
seen.add(candidate);
return true;
});
}, [rawUrl, url]);
const activeUrl = sources[sourceIndex] || '';
useEffect(() => {
setSourceIndex(0);
setMediaError(false);
setMediaLoaded(false);
setPaused(false);
}, [rawUrl, url]);
useEffect(() => {
setMediaLoaded(false);
}, [activeUrl]);
const handleMediaFailure = useCallback(() => {
setSourceIndex((idx) => {
const next = idx + 1;
if (next < sources.length) {
setMediaError(false);
return next;
}
setMediaError(true);
return idx;
});
}, [sources.length]);
const handleMediaReady = useCallback(() => {
setMediaLoaded(true);
}, []);
useEffect(() => {
if (sourceIndex !== 0 || sources.length < 2 || mediaLoaded || mediaError) return;
const timeoutMs = mediaType === 'hls' ? 3200 : 1800;
const timer = window.setTimeout(() => {
setSourceIndex((idx) => {
if (idx !== 0 || mediaLoaded) return idx;
return 1;
});
}, timeoutMs);
return () => window.clearTimeout(timer);
}, [mediaError, mediaLoaded, mediaType, sourceIndex, sources.length]);
const togglePlay = useCallback(() => { const togglePlay = useCallback(() => {
if (mediaType === 'hls') { if (mediaType === 'hls') {
@@ -176,17 +230,21 @@ export function CctvFullscreenModal({
overflow: 'hidden', overflow: 'hidden',
}} }}
> >
{url ? ( {activeUrl ? (
<> <>
{mediaType === 'video' && !mediaError && ( {mediaType === 'video' && !mediaError && (
<video <video
key={activeUrl}
ref={videoRef} ref={videoRef}
src={url} src={activeUrl}
autoPlay autoPlay
loop loop
muted muted
playsInline playsInline
onError={() => setMediaError(true)} onError={handleMediaFailure}
onCanPlay={handleMediaReady}
onLoadedData={handleMediaReady}
onPlaying={handleMediaReady}
style={{ style={{
maxWidth: '100%', maxWidth: '100%',
maxHeight: 'calc(100vh - 260px)', maxHeight: 'calc(100vh - 260px)',
@@ -197,15 +255,18 @@ export function CctvFullscreenModal({
)} )}
{mediaType === 'hls' && !mediaError && ( {mediaType === 'hls' && !mediaError && (
<HlsVideo <HlsVideo
key={activeUrl}
ref={hlsRef} ref={hlsRef}
url={url} url={activeUrl}
onError={() => setMediaError(true)} onError={handleMediaFailure}
className="" onLoaded={handleMediaReady}
className="max-w-full max-h-[calc(100vh-260px)] object-contain"
/> />
)} )}
{mediaType === 'mjpeg' && ( {mediaType === 'mjpeg' && (
<img <img
src={url} key={activeUrl}
src={activeUrl}
alt="MJPEG Feed" alt="MJPEG Feed"
style={{ style={{
maxWidth: '100%', maxWidth: '100%',
@@ -213,14 +274,14 @@ export function CctvFullscreenModal({
objectFit: 'contain', objectFit: 'contain',
filter: 'contrast(1.25) saturate(0.5)', filter: 'contrast(1.25) saturate(0.5)',
}} }}
onError={(e) => { onError={handleMediaFailure}
(e.target as HTMLImageElement).style.display = 'none'; onLoad={handleMediaReady}
}}
/> />
)} )}
{(mediaType === 'image' || mediaType === 'satellite') && ( {(mediaType === 'image' || mediaType === 'satellite') && (
<img <img
src={url} key={activeUrl}
src={activeUrl}
alt="CCTV Feed" alt="CCTV Feed"
style={{ style={{
maxWidth: '100%', maxWidth: '100%',
@@ -228,10 +289,8 @@ export function CctvFullscreenModal({
objectFit: 'contain', objectFit: 'contain',
filter: 'contrast(1.25) saturate(0.5)', filter: 'contrast(1.25) saturate(0.5)',
}} }}
onError={(e) => { onError={handleMediaFailure}
const target = e.target as HTMLImageElement; onLoad={handleMediaReady}
target.style.display = 'none';
}}
/> />
)} )}
@@ -239,7 +298,7 @@ export function CctvFullscreenModal({
{mediaError && ( {mediaError && (
<div style={{ fontSize: 11, color: 'rgba(239,68,68,0.7)', fontFamily: 'monospace', letterSpacing: '0.15em', textAlign: 'center', padding: 40 }}> <div style={{ fontSize: 11, color: 'rgba(239,68,68,0.7)', fontFamily: 'monospace', letterSpacing: '0.15em', textAlign: 'center', padding: 40 }}>
FEED UNAVAILABLE<br /> FEED UNAVAILABLE<br />
<span style={{ fontSize: 9, color: 'rgba(148,163,184,0.5)' }}>stream failed to load source may be offline</span> <span style={{ fontSize: 9, color: 'rgba(148,163,184,0.5)' }}>proxy and direct source both failed</span>
</div> </div>
)} )}
@@ -329,10 +388,10 @@ export function CctvFullscreenModal({
{cameraName} {cameraName}
</span> </span>
<div style={{ display: 'flex', gap: 10 }}> <div style={{ display: 'flex', gap: 10 }}>
{url && ( {activeUrl && (
<> <>
<a <a
href={url} href={rawUrl || activeUrl}
target="_blank" target="_blank"
rel="noopener noreferrer" rel="noopener noreferrer"
style={{ style={{
@@ -354,7 +413,7 @@ export function CctvFullscreenModal({
<button <button
onClick={async () => { onClick={async () => {
try { try {
await navigator.clipboard.writeText(url); await navigator.clipboard.writeText(rawUrl || activeUrl);
} catch { /* ignore */ } } catch { /* ignore */ }
}} }}
style={{ style={{
+240 -28
View File
@@ -111,14 +111,27 @@ const MeshChat = React.memo(function MeshChat(props: MeshChatProps) {
identityWizardStatus, identityWizardStatus,
setIdentityWizardStatus, setIdentityWizardStatus,
meshQuickStatus, meshQuickStatus,
meshSessionActive,
publicMeshAddress, publicMeshAddress,
meshView, meshView,
setMeshView, setMeshView,
meshDirectTarget, meshDirectTarget,
setMeshDirectTarget, setMeshDirectTarget,
meshMqttSettings,
meshMqttForm,
setMeshMqttForm,
meshMqttBusy,
meshMqttStatusText,
meshMqttEnabled,
meshMqttRunning,
meshMqttConnected,
meshMqttConnectionLabel,
saveMeshMqttSettings,
refreshMeshMqttSettings,
// Identity // Identity
identity, identity,
publicIdentity, publicIdentity,
hasStoredPublicLaneIdentity,
hasPublicLaneIdentity, hasPublicLaneIdentity,
hasId, hasId,
shouldShowIdentityWarning, shouldShowIdentityWarning,
@@ -255,6 +268,7 @@ const MeshChat = React.memo(function MeshChat(props: MeshChatProps) {
openChat, openChat,
handleCreatePublicIdentity, handleCreatePublicIdentity,
handleQuickCreatePublicIdentity, handleQuickCreatePublicIdentity,
handleActivatePublicMeshSession,
handleLeaveWormholeForPublicMesh, handleLeaveWormholeForPublicMesh,
handleResetPublicIdentity, handleResetPublicIdentity,
handleBootstrapPrivateIdentity, handleBootstrapPrivateIdentity,
@@ -324,6 +338,40 @@ const MeshChat = React.memo(function MeshChat(props: MeshChatProps) {
} }
void handleRequestAccess(targetId); void handleRequestAccess(targetId);
}; };
const meshActivationText =
meshQuickStatus?.text ||
(publicMeshBlockedByWormhole
? hasStoredPublicLaneIdentity
? 'Wormhole is active. Turning MeshChat on will turn Wormhole off and use your saved public mesh key.'
: 'Wormhole is active. Turning MeshChat on will turn Wormhole off and mint a separate public mesh key.'
: hasStoredPublicLaneIdentity
? 'MeshChat is off. Turn it on to use your saved public mesh key.'
: 'Public mesh posting needs a mesh key. One tap gets you a fresh address.');
const handleMeshActivationAction = () => {
if (hasStoredPublicLaneIdentity) {
void handleActivatePublicMeshSession();
return;
}
if (publicMeshBlockedByWormhole) {
void handleLeaveWormholeForPublicMesh();
return;
}
void handleQuickCreatePublicIdentity();
};
const meshActivationLabel = identityWizardBusy
? 'GETTING MESH KEY'
: hasStoredPublicLaneIdentity
? 'TURN ON MESH'
: publicMeshBlockedByWormhole
? 'TURN OFF WORMHOLE FOR MESH'
: 'GET MESH KEY';
const meshActivationSideLabel = identityWizardBusy
? 'WORKING...'
: hasStoredPublicLaneIdentity
? 'USE SAVED KEY'
: publicMeshBlockedByWormhole
? 'AUTO DISABLE'
: 'ONE TAP';
return ( return (
<div <div
@@ -1118,18 +1166,189 @@ const MeshChat = React.memo(function MeshChat(props: MeshChatProps) {
> >
INBOX INBOX
</button> </button>
<button
onClick={() => setMeshView('settings')}
className={`px-2 py-0.5 text-[11px] font-mono tracking-wider border transition-colors ${
meshView === 'settings'
? 'border-cyan-500/40 text-cyan-300 bg-cyan-950/20'
: 'border-[var(--border-primary)]/40 text-[var(--text-muted)] hover:text-cyan-300'
}`}
>
SETTINGS
</button>
</div> </div>
<div className="text-[10px] font-mono text-[var(--text-muted)] truncate"> <div
{publicMeshAddress ? `ADDR ${publicMeshAddress.toUpperCase()}` : 'NO PUBLIC MESH ADDRESS'} className={`text-[10px] font-mono truncate ${
meshMqttConnected
? 'text-green-300/80'
: meshMqttEnabled
? 'text-amber-300/80'
: 'text-[var(--text-muted)]'
}`}
>
{meshSessionActive && publicMeshAddress
? `${meshMqttConnectionLabel} / ADDR ${publicMeshAddress.toUpperCase()}`
: publicMeshAddress
? `${meshMqttConnectionLabel} / KEY SAVED`
: `${meshMqttConnectionLabel} / NO ADDRESS`}
</div> </div>
</div> </div>
<div className="flex-1 overflow-y-auto styled-scrollbar px-3 py-1.5 border-l-2 border-cyan-800/25"> <div className="flex-1 overflow-y-auto styled-scrollbar px-3 py-1.5 border-l-2 border-cyan-800/25">
{meshView === 'channel' && filteredMeshMessages.length === 0 && ( {meshView === 'settings' && (
<div className="space-y-2 py-1 text-[11px] font-mono">
<div className="border border-cyan-800/35 bg-cyan-950/10 p-2">
<div className="flex items-center justify-between gap-2">
<div>
<div className="text-cyan-300 tracking-[0.18em]">MESHTASTIC MQTT</div>
<div className="mt-1 text-[10px] text-[var(--text-muted)] leading-[1.5]">
Public Mesh is separate from Wormhole. Turning MQTT on disables the private Wormhole lane for MeshChat.
</div>
</div>
<span
className={`shrink-0 border px-2 py-1 text-[10px] tracking-[0.16em] ${
meshMqttConnected
? 'border-green-500/40 text-green-300'
: meshMqttEnabled
? 'border-amber-500/40 text-amber-300'
: 'border-red-500/35 text-red-300'
}`}
>
{meshMqttConnectionLabel}
</span>
</div>
{meshMqttSettings?.runtime?.last_error && (
<div className="mt-2 text-red-300/80">
LAST ERROR: {meshMqttSettings.runtime.last_error}
</div>
)}
{meshMqttRunning && !meshMqttConnected && !meshMqttSettings?.runtime?.last_error && (
<div className="mt-2 text-amber-300/80">
MQTT bridge is starting. Live messages appear after broker connect.
</div>
)}
</div>
<div className="grid grid-cols-[1fr_70px] gap-2">
<label className="space-y-1">
<span className="text-[var(--text-muted)]">BROKER</span>
<input
value={meshMqttForm.broker}
onChange={(e) => setMeshMqttForm((prev) => ({ ...prev, broker: e.target.value }))}
className="w-full border border-[var(--border-primary)] bg-black/30 px-2 py-1 text-cyan-200 outline-none focus:border-cyan-500/50"
/>
</label>
<label className="space-y-1">
<span className="text-[var(--text-muted)]">PORT</span>
<input
value={meshMqttForm.port}
onChange={(e) => setMeshMqttForm((prev) => ({ ...prev, port: e.target.value }))}
className="w-full border border-[var(--border-primary)] bg-black/30 px-2 py-1 text-cyan-200 outline-none focus:border-cyan-500/50"
/>
</label>
</div>
<label className="block space-y-1">
<span className="text-[var(--text-muted)]">BROKER LOGIN (optional)</span>
<input
value={meshMqttForm.username}
onChange={(e) => setMeshMqttForm((prev) => ({ ...prev, username: e.target.value }))}
placeholder="blank uses public Meshtastic default"
className="w-full border border-[var(--border-primary)] bg-black/30 px-2 py-1 text-cyan-200 outline-none focus:border-cyan-500/50"
/>
</label>
<label className="block space-y-1">
<span className="text-[var(--text-muted)]">
BROKER PASSWORD {meshMqttSettings?.uses_default_credentials ? '(public default)' : meshMqttSettings?.has_password ? '(saved)' : ''}
</span>
<input
type="password"
value={meshMqttForm.password}
onChange={(e) => setMeshMqttForm((prev) => ({ ...prev, password: e.target.value }))}
placeholder={
meshMqttSettings?.uses_default_credentials
? 'blank uses public Meshtastic default'
: meshMqttSettings?.has_password
? 'leave blank to keep saved password'
: 'blank uses public Meshtastic default'
}
className="w-full border border-[var(--border-primary)] bg-black/30 px-2 py-1 text-cyan-200 outline-none placeholder:text-[var(--text-muted)] focus:border-cyan-500/50"
/>
</label>
<label className="block space-y-1">
<span className="text-[var(--text-muted)]">
CHANNEL PSK HEX {meshMqttSettings?.has_psk ? '(saved)' : '(default LongFast if blank)'}
</span>
<input
type="password"
value={meshMqttForm.psk}
onChange={(e) => setMeshMqttForm((prev) => ({ ...prev, psk: e.target.value }))}
placeholder="blank uses default LongFast key"
className="w-full border border-[var(--border-primary)] bg-black/30 px-2 py-1 text-cyan-200 outline-none placeholder:text-[var(--text-muted)] focus:border-cyan-500/50"
/>
</label>
<label className="flex items-center gap-2 border border-[var(--border-primary)]/40 bg-black/20 px-2 py-1 text-cyan-200">
<input
type="checkbox"
checked={meshMqttForm.include_default_roots}
onChange={(e) =>
setMeshMqttForm((prev) => ({ ...prev, include_default_roots: e.target.checked }))
}
/>
DEFAULT PUBLIC ROOTS
</label>
<label className="block space-y-1">
<span className="text-[var(--text-muted)]">EXTRA ROOTS</span>
<input
value={meshMqttForm.extra_roots}
onChange={(e) => setMeshMqttForm((prev) => ({ ...prev, extra_roots: e.target.value }))}
placeholder="comma separated, optional"
className="w-full border border-[var(--border-primary)] bg-black/30 px-2 py-1 text-cyan-200 outline-none placeholder:text-[var(--text-muted)] focus:border-cyan-500/50"
/>
</label>
<div className="grid grid-cols-3 gap-2 pt-1">
<button
onClick={() => void saveMeshMqttSettings({ enabled: true })}
disabled={meshMqttBusy}
className="border border-green-600/40 bg-green-950/20 px-2 py-1.5 text-green-300 hover:bg-green-950/35 disabled:opacity-50"
>
ENABLE
</button>
<button
onClick={() => void saveMeshMqttSettings({ enabled: false })}
disabled={meshMqttBusy}
className="border border-red-600/35 bg-red-950/15 px-2 py-1.5 text-red-300 hover:bg-red-950/25 disabled:opacity-50"
>
DISABLE
</button>
<button
onClick={() => void refreshMeshMqttSettings()}
disabled={meshMqttBusy}
className="border border-cyan-700/40 bg-cyan-950/15 px-2 py-1.5 text-cyan-300 hover:bg-cyan-950/25 disabled:opacity-50"
>
REFRESH
</button>
</div>
{meshMqttStatusText && (
<div className="text-[10px] text-cyan-200/80 leading-[1.5]">{meshMqttStatusText}</div>
)}
</div>
)}
{!meshSessionActive && meshView !== 'settings' && (
<div className="text-[12px] font-mono text-green-300/70 text-center py-4 leading-[1.65]">
MeshChat is off. Turn it on to connect the public mesh lane.
</div>
)}
{meshSessionActive && meshView === 'channel' && filteredMeshMessages.length === 0 && (
<div className="text-[12px] font-mono text-[var(--text-muted)] text-center py-4 leading-[1.65]"> <div className="text-[12px] font-mono text-[var(--text-muted)] text-center py-4 leading-[1.65]">
No messages from {meshRegion} / {meshChannel} No messages from {meshRegion} / {meshChannel}
</div> </div>
)} )}
{meshView === 'inbox' && ( {meshSessionActive && meshView === 'inbox' && (
<> <>
{!publicMeshAddress && ( {!publicMeshAddress && (
<div className="text-[12px] font-mono text-[var(--text-muted)] text-center py-4 leading-[1.65]"> <div className="text-[12px] font-mono text-[var(--text-muted)] text-center py-4 leading-[1.65]">
@@ -2049,7 +2268,9 @@ const MeshChat = React.memo(function MeshChat(props: MeshChatProps) {
? meshDirectTarget ? meshDirectTarget
? `→ MESH / TO ${meshDirectTarget.toUpperCase()}` ? `→ MESH / TO ${meshDirectTarget.toUpperCase()}`
: `→ MESH / ${meshRegion} / ${meshChannel}` : `→ MESH / ${meshRegion} / ${meshChannel}`
: '→ MESH LOCKED' : hasStoredPublicLaneIdentity
? '→ MESH OFF'
: '→ MESH LOCKED'
: activeTab === 'dms' && secureDmBlocked : activeTab === 'dms' && secureDmBlocked
? '→ DEAD DROP LOCKED' ? '→ DEAD DROP LOCKED'
: dmView === 'chat' && selectedContact : dmView === 'chat' && selectedContact
@@ -2068,10 +2289,7 @@ const MeshChat = React.memo(function MeshChat(props: MeshChatProps) {
: 'text-green-300/70' : 'text-green-300/70'
}`} }`}
> >
{meshQuickStatus?.text || {meshActivationText}
(publicMeshBlockedByWormhole
? 'Wormhole is active. Turn it off here and we will mint a separate public mesh key for you.'
: 'Public mesh posting needs a mesh key. One tap gets you a fresh address.')}
</div> </div>
)} )}
<div className="flex items-center gap-2 px-3 pb-2 pt-1"> <div className="flex items-center gap-2 px-3 pb-2 pt-1">
@@ -2103,30 +2321,16 @@ const MeshChat = React.memo(function MeshChat(props: MeshChatProps) {
</button> </button>
) : activeTab === 'meshtastic' && !hasPublicLaneIdentity ? ( ) : activeTab === 'meshtastic' && !hasPublicLaneIdentity ? (
<button <button
onClick={() => { onClick={handleMeshActivationAction}
if (publicMeshBlockedByWormhole) {
void handleLeaveWormholeForPublicMesh();
return;
}
void handleQuickCreatePublicIdentity();
}}
disabled={identityWizardBusy} disabled={identityWizardBusy}
className="w-full flex items-center justify-between gap-2 px-3 py-2 border border-green-700/40 bg-green-950/15 text-green-300 hover:bg-green-950/25 hover:border-green-500/50 transition-colors" className="w-full flex items-center justify-between gap-2 px-3 py-2 border border-green-700/40 bg-green-950/15 text-green-300 hover:bg-green-950/25 hover:border-green-500/50 transition-colors"
> >
<span className="inline-flex items-center gap-2 text-sm font-mono tracking-[0.2em]"> <span className="inline-flex items-center gap-2 text-sm font-mono tracking-[0.2em]">
<Radio size={11} /> <Radio size={11} />
{identityWizardBusy {meshActivationLabel}
? 'GETTING MESH KEY'
: publicMeshBlockedByWormhole
? 'TURN OFF WORMHOLE FOR MESH'
: 'GET MESH KEY'}
</span> </span>
<span className="text-[12px] font-mono text-green-300/70"> <span className="text-[12px] font-mono text-green-300/70">
{identityWizardBusy {meshActivationSideLabel}
? 'WORKING...'
: publicMeshBlockedByWormhole
? 'AUTO FIX'
: 'ONE TAP'}
</span> </span>
</button> </button>
) : activeTab === 'meshtastic' && meshDirectTarget ? ( ) : activeTab === 'meshtastic' && meshDirectTarget ? (
@@ -2375,8 +2579,8 @@ const MeshChat = React.memo(function MeshChat(props: MeshChatProps) {
CURRENT STATE CURRENT STATE
</div> </div>
<div className="grid grid-cols-1 gap-1 text-[13px] font-mono text-[var(--text-secondary)] leading-[1.5]"> <div className="grid grid-cols-1 gap-1 text-[13px] font-mono text-[var(--text-secondary)] leading-[1.5]">
<div>Public mesh key: {hasPublicLaneIdentity ? 'active' : 'not issued'}</div> <div>Public mesh key: {hasPublicLaneIdentity ? 'active' : hasStoredPublicLaneIdentity ? 'saved / off' : 'not issued'}</div>
<div>Public mesh address: {hasPublicLaneIdentity && publicMeshAddress ? publicMeshAddress.toUpperCase() : 'not ready'}</div> <div>Public mesh address: {publicMeshAddress ? publicMeshAddress.toUpperCase() : 'not ready'}</div>
<div>Wormhole lane: {wormholeEnabled && wormholeReadyState ? 'active' : wormholeEnabled ? 'starting' : 'off'}</div> <div>Wormhole lane: {wormholeEnabled && wormholeReadyState ? 'active' : wormholeEnabled ? 'starting' : 'off'}</div>
<div>Wormhole descriptor: {wormholeDescriptor?.nodeId || 'not cached yet'}</div> <div>Wormhole descriptor: {wormholeDescriptor?.nodeId || 'not cached yet'}</div>
</div> </div>
@@ -2385,6 +2589,10 @@ const MeshChat = React.memo(function MeshChat(props: MeshChatProps) {
<div className="grid grid-cols-1 gap-2"> <div className="grid grid-cols-1 gap-2">
<button <button
onClick={() => { onClick={() => {
if (hasStoredPublicLaneIdentity) {
void handleActivatePublicMeshSession();
return;
}
if (publicMeshBlockedByWormhole) { if (publicMeshBlockedByWormhole) {
void handleLeaveWormholeForPublicMesh(); void handleLeaveWormholeForPublicMesh();
return; return;
@@ -2396,12 +2604,16 @@ const MeshChat = React.memo(function MeshChat(props: MeshChatProps) {
> >
{hasPublicLaneIdentity {hasPublicLaneIdentity
? 'MESH KEY ACTIVE' ? 'MESH KEY ACTIVE'
: hasStoredPublicLaneIdentity
? 'TURN ON MESH'
: publicMeshBlockedByWormhole : publicMeshBlockedByWormhole
? 'TURN OFF WORMHOLE FOR MESH' ? 'TURN OFF WORMHOLE FOR MESH'
: 'GET MESH KEY'} : 'GET MESH KEY'}
<div className="mt-1 text-[13px] text-green-200/70 normal-case tracking-normal leading-[1.45]"> <div className="mt-1 text-[13px] text-green-200/70 normal-case tracking-normal leading-[1.45]">
{hasPublicLaneIdentity {hasPublicLaneIdentity
? 'Your public mesh key is already live for posting.' ? 'Your public mesh key is already live for posting.'
: hasStoredPublicLaneIdentity
? 'Use your saved public mesh key. This turns Wormhole off first if it is active.'
: publicMeshBlockedByWormhole : publicMeshBlockedByWormhole
? 'One tap turns Wormhole off and mints a separate public mesh key.' ? 'One tap turns Wormhole off and mints a separate public mesh key.'
: 'One tap for a working mesh key and address.'} : 'One tap for a working mesh key and address.'}
@@ -15,9 +15,6 @@ import {
import type { DesktopControlAuditReport } from '@/lib/desktopControlContract'; import type { DesktopControlAuditReport } from '@/lib/desktopControlContract';
import { fetchPrivacyProfileSnapshot } from '@/mesh/controlPlaneStatusClient'; import { fetchPrivacyProfileSnapshot } from '@/mesh/controlPlaneStatusClient';
import { import {
clearBrowserIdentityState,
derivePublicMeshAddress,
generateNodeKeys,
getNodeIdentity, getNodeIdentity,
getStoredNodeDescriptor, getStoredNodeDescriptor,
getWormholeIdentityDescriptor, getWormholeIdentityDescriptor,
@@ -31,9 +28,7 @@ import {
updateContact, updateContact,
blockContact, blockContact,
getDMNotify, getDMNotify,
getPublicKeyAlgo,
nextSequence, nextSequence,
signEvent,
verifyEventSignature, verifyEventSignature,
verifyRawSignature, verifyRawSignature,
purgeBrowserContactGraph, purgeBrowserContactGraph,
@@ -130,7 +125,6 @@ import {
preferredDmPeerId, preferredDmPeerId,
} from '@/mesh/meshDmConsent'; } from '@/mesh/meshDmConsent';
import { deriveSasPhrase } from '@/mesh/meshSas'; import { deriveSasPhrase } from '@/mesh/meshSas';
import { PROTOCOL_VERSION } from '@/mesh/meshProtocol';
import { validateEventPayload } from '@/mesh/meshSchema'; import { validateEventPayload } from '@/mesh/meshSchema';
import { import {
buildDmTrustHint, buildDmTrustHint,
@@ -223,6 +217,94 @@ interface GateCompatConsentPromptState {
reason: string; reason: string;
} }
interface MeshMqttRuntime {
enabled?: boolean;
running?: boolean;
connected?: boolean;
broker?: string;
port?: number;
username?: string;
client_id?: string;
message_log_size?: number;
signal_log_size?: number;
last_error?: string;
last_connected_at?: number;
last_disconnected_at?: number;
}
interface MeshMqttSettings {
enabled: boolean;
broker: string;
port: number;
username: string;
uses_default_credentials?: boolean;
has_password: boolean;
has_psk: boolean;
include_default_roots: boolean;
extra_roots: string;
extra_topics: string;
runtime?: MeshMqttRuntime;
}
interface MeshMqttForm {
broker: string;
port: string;
username: string;
password: string;
psk: string;
include_default_roots: boolean;
extra_roots: string;
extra_topics: string;
}
const PUBLIC_MESH_ADDRESS_KEY = 'sb_public_meshtastic_address';
function normalizePublicMeshAddress(value: string): string {
const raw = String(value || '').trim().toLowerCase();
const body = raw.startsWith('!') ? raw.slice(1) : raw;
if (!/^[0-9a-f]{8}$/.test(body)) return '';
return `!${body}`;
}
function readStoredPublicMeshAddress(): string {
if (typeof window === 'undefined') return '';
try {
return normalizePublicMeshAddress(window.localStorage.getItem(PUBLIC_MESH_ADDRESS_KEY) || '');
} catch {
return '';
}
}
function writeStoredPublicMeshAddress(address: string): void {
if (typeof window === 'undefined') return;
const normalized = normalizePublicMeshAddress(address);
if (!normalized) return;
try {
window.localStorage.setItem(PUBLIC_MESH_ADDRESS_KEY, normalized);
} catch {
/* ignore */
}
}
function clearStoredPublicMeshAddress(): void {
if (typeof window === 'undefined') return;
try {
window.localStorage.removeItem(PUBLIC_MESH_ADDRESS_KEY);
} catch {
/* ignore */
}
}
function createPublicMeshAddress(): string {
if (typeof window !== 'undefined' && window.crypto?.getRandomValues) {
const value = new Uint32Array(1);
window.crypto.getRandomValues(value);
if (value[0]) return `!${value[0].toString(16).padStart(8, '0')}`;
}
const fallback = Math.floor((Date.now() ^ Math.floor(Math.random() * 0xffffffff)) >>> 0);
return `!${fallback.toString(16).padStart(8, '0')}`;
}
function describeGateCompatConsentRequired(): string { function describeGateCompatConsentRequired(): string {
return 'Local gate runtime is unavailable for this room.'; return 'Local gate runtime is unavailable for this room.';
} }
@@ -313,9 +395,23 @@ export function useMeshChatController({
const [identityWizardBusy, setIdentityWizardBusy] = useState(false); const [identityWizardBusy, setIdentityWizardBusy] = useState(false);
const [identityWizardStatus, setIdentityWizardStatus] = useState<{ type: 'ok' | 'err'; text: string } | null>(null); const [identityWizardStatus, setIdentityWizardStatus] = useState<{ type: 'ok' | 'err'; text: string } | null>(null);
const [meshQuickStatus, setMeshQuickStatus] = useState<{ type: 'ok' | 'err'; text: string } | null>(null); const [meshQuickStatus, setMeshQuickStatus] = useState<{ type: 'ok' | 'err'; text: string } | null>(null);
const [meshSessionActive, setMeshSessionActive] = useState(false);
const [publicMeshAddress, setPublicMeshAddress] = useState(''); const [publicMeshAddress, setPublicMeshAddress] = useState('');
const [meshView, setMeshView] = useState<'channel' | 'inbox'>('channel'); const [meshView, setMeshView] = useState<'channel' | 'inbox' | 'settings'>('channel');
const [meshDirectTarget, setMeshDirectTarget] = useState(''); const [meshDirectTarget, setMeshDirectTarget] = useState('');
const [meshMqttSettings, setMeshMqttSettings] = useState<MeshMqttSettings | null>(null);
const [meshMqttForm, setMeshMqttForm] = useState<MeshMqttForm>({
broker: 'mqtt.meshtastic.org',
port: '1883',
username: '',
password: '',
psk: '',
include_default_roots: true,
extra_roots: '',
extra_topics: '',
});
const [meshMqttBusy, setMeshMqttBusy] = useState(false);
const [meshMqttStatusText, setMeshMqttStatusText] = useState('');
// Identity // Identity
const [identity, setIdentity] = useState<NodeIdentity | null>(null); const [identity, setIdentity] = useState<NodeIdentity | null>(null);
@@ -328,13 +424,123 @@ export function useMeshChatController({
const [recentPrivateFallbackReason, setRecentPrivateFallbackReason] = useState(''); const [recentPrivateFallbackReason, setRecentPrivateFallbackReason] = useState('');
const [unresolvedSenderSealCount, setUnresolvedSenderSealCount] = useState(0); const [unresolvedSenderSealCount, setUnresolvedSenderSealCount] = useState(0);
const [privacyProfile, setPrivacyProfile] = useState<'default' | 'high'>('default'); const [privacyProfile, setPrivacyProfile] = useState<'default' | 'high'>('default');
const publicIdentity = clientHydrated ? getNodeIdentity() : null; const storedPublicMeshAddress = clientHydrated ? readStoredPublicMeshAddress() : '';
const hasPublicLaneIdentity = clientHydrated && Boolean(publicIdentity) && hasSovereignty(); const hasStoredPublicLaneIdentity = clientHydrated && Boolean(storedPublicMeshAddress);
const publicIdentity = null;
const hasPublicLaneIdentity = meshSessionActive && Boolean(publicMeshAddress);
const hasId = Boolean(identity) && (hasSovereignty() || wormholeEnabled); const hasId = Boolean(identity) && (hasSovereignty() || wormholeEnabled);
const shouldShowIdentityWarning = activeTab !== 'meshtastic' && !hasId; const shouldShowIdentityWarning = activeTab !== 'meshtastic' && !hasId;
const privateInfonetReady = wormholeEnabled && wormholeReadyState; const privateInfonetReady = wormholeEnabled && wormholeReadyState;
const publicMeshBlockedByWormhole = wormholeEnabled && wormholeReadyState && !hasPublicLaneIdentity; const publicMeshBlockedByWormhole = wormholeEnabled || wormholeReadyState;
const dmSendQueue = useRef<(() => Promise<void>)[]>([]); const dmSendQueue = useRef<(() => Promise<void>)[]>([]);
const meshMqttRuntime = meshMqttSettings?.runtime;
const meshMqttEnabled = Boolean(meshMqttSettings?.enabled || meshMqttRuntime?.enabled);
const meshMqttRunning = Boolean(meshMqttRuntime?.running);
const meshMqttConnected = Boolean(meshMqttRuntime?.connected);
const meshMqttConnectionLabel = !meshMqttEnabled
? 'MQTT OFF'
: meshMqttConnected
? 'MQTT LIVE'
: meshMqttRunning
? 'MQTT CONNECTING'
: 'MQTT STARTING';
const applyMeshMqttSettings = useCallback((data: MeshMqttSettings) => {
setMeshMqttSettings(data);
setMeshMqttForm((prev) => ({
broker: data.broker || prev.broker || 'mqtt.meshtastic.org',
port: String(data.port || prev.port || '1883'),
username: data.uses_default_credentials ? '' : data.username || prev.username || '',
password: '',
psk: '',
include_default_roots: Boolean(data.include_default_roots),
extra_roots: data.extra_roots || '',
extra_topics: data.extra_topics || '',
}));
}, []);
const refreshMeshMqttSettings = useCallback(async () => {
try {
const res = await fetch(`${API_BASE}/api/settings/meshtastic-mqtt`, { cache: 'no-store' });
if (!res.ok) return null;
const data = (await res.json()) as MeshMqttSettings;
applyMeshMqttSettings(data);
return data;
} catch {
return null;
}
}, [applyMeshMqttSettings]);
const saveMeshMqttSettings = useCallback(
async (updates: Partial<MeshMqttForm> & { enabled?: boolean } = {}) => {
setMeshMqttBusy(true);
setMeshMqttStatusText('');
try {
const nextForm = { ...meshMqttForm, ...updates };
const body: Record<string, unknown> = {
broker: nextForm.broker.trim() || 'mqtt.meshtastic.org',
port: Number.parseInt(nextForm.port, 10) || 1883,
username: nextForm.username.trim(),
include_default_roots: Boolean(nextForm.include_default_roots),
extra_roots: nextForm.extra_roots.trim(),
extra_topics: nextForm.extra_topics.trim(),
};
if (!nextForm.username.trim() && !nextForm.password.trim()) {
body.password = '';
}
if (typeof updates.enabled === 'boolean') {
body.enabled = updates.enabled;
}
if (nextForm.password.trim()) {
body.password = nextForm.password;
}
if (nextForm.psk.trim()) {
body.psk = nextForm.psk.trim();
}
const res = await fetch(`${API_BASE}/api/settings/meshtastic-mqtt`, {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
});
if (!res.ok) {
const detail = await res.text().catch(() => '');
throw new Error(detail || `HTTP ${res.status}`);
}
const data = (await res.json()) as MeshMqttSettings;
applyMeshMqttSettings(data);
if (data.enabled) {
setWormholeEnabled(false);
setWormholeReadyState(false);
setWormholeRnsReady(false);
setWormholeRnsDirectReady(false);
setWormholeRnsPeers({ active: 0, configured: 0 });
setSecureModeCached(false);
}
const status = data.runtime?.connected
? 'MQTT bridge connected.'
: data.enabled
? 'MQTT bridge enabled. Connection may take a few seconds.'
: 'MQTT bridge disabled.';
setMeshMqttStatusText(status);
return { ok: true as const, text: status, data };
} catch (err) {
const text = err instanceof Error ? err.message : 'MQTT settings update failed';
setMeshMqttStatusText(text);
return { ok: false as const, text };
} finally {
setMeshMqttBusy(false);
}
},
[applyMeshMqttSettings, meshMqttForm],
);
const enableMeshMqttBridge = useCallback(async () => {
const result = await saveMeshMqttSettings({ enabled: true });
if (!result.ok) {
throw new Error(result.text);
}
return result;
}, [saveMeshMqttSettings]);
const dmSendTimer = useRef<ReturnType<typeof setTimeout> | null>(null); const dmSendTimer = useRef<ReturnType<typeof setTimeout> | null>(null);
const streamEnabledForSelectedGateRef = useRef(false); const streamEnabledForSelectedGateRef = useRef(false);
const displayPublicMeshSender = useCallback( const displayPublicMeshSender = useCallback(
@@ -342,15 +548,14 @@ export function useMeshChatController({
if (!sender) return '???'; if (!sender) return '???';
if ( if (
hasPublicLaneIdentity && hasPublicLaneIdentity &&
publicIdentity?.nodeId &&
publicMeshAddress && publicMeshAddress &&
sender.toLowerCase() === publicIdentity.nodeId.toLowerCase() sender.toLowerCase() === publicMeshAddress.toLowerCase()
) { ) {
return publicMeshAddress.toUpperCase(); return publicMeshAddress.toUpperCase();
} }
return sender; return sender;
}, },
[hasPublicLaneIdentity, publicIdentity?.nodeId, publicMeshAddress], [hasPublicLaneIdentity, publicMeshAddress],
); );
const openIdentityWizard = useCallback( const openIdentityWizard = useCallback(
@@ -365,6 +570,14 @@ export function useMeshChatController({
setClientHydrated(true); setClientHydrated(true);
}, []); }, []);
useEffect(() => {
if (!clientHydrated) return;
setPublicMeshAddress(readStoredPublicMeshAddress());
setMeshSessionActive(false);
setMeshMessages([]);
setMeshQuickStatus(null);
}, [clientHydrated]);
useEffect( useEffect(
() => () =>
subscribeGateSessionStreamStatus((nextStatus) => { subscribeGateSessionStreamStatus((nextStatus) => {
@@ -450,6 +663,8 @@ export function useMeshChatController({
setSecureModeCached(enabled); setSecureModeCached(enabled);
setWormholeEnabled(enabled); setWormholeEnabled(enabled);
if (enabled) { if (enabled) {
setMeshSessionActive(false);
setMeshMessages([]);
purgeBrowserContactGraph(); purgeBrowserContactGraph();
void hydrateWormholeContacts(); void hydrateWormholeContacts();
} }
@@ -513,25 +728,6 @@ export function useMeshChatController({
}; };
}, []); }, []);
useEffect(() => {
let alive = true;
const senderId = publicIdentity?.nodeId || '';
if (!senderId || !globalThis.crypto?.subtle) {
setPublicMeshAddress('');
return;
}
derivePublicMeshAddress(senderId)
.then((addr) => {
if (alive) setPublicMeshAddress(addr);
})
.catch(() => {
if (alive) setPublicMeshAddress('');
});
return () => {
alive = false;
};
}, [publicIdentity?.nodeId]);
const flushDmQueue = useCallback(async () => { const flushDmQueue = useCallback(async () => {
const queue = dmSendQueue.current.splice(0); const queue = dmSendQueue.current.splice(0);
if (dmSendTimer.current) { if (dmSendTimer.current) {
@@ -1138,10 +1334,40 @@ export function useMeshChatController({
[meshMessages, mutedUsers], [meshMessages, mutedUsers],
); );
const meshInboxMessages = useMemo(() => { const meshInboxMessages = useMemo(() => {
if (!publicMeshAddress) return []; if (!meshSessionActive || !publicMeshAddress) return [];
const target = publicMeshAddress.toLowerCase(); const target = publicMeshAddress.toLowerCase();
return filteredMeshMessages.filter((m) => String(m.to || '').toLowerCase() === target); return filteredMeshMessages.filter((m) => String(m.to || '').toLowerCase() === target);
}, [filteredMeshMessages, publicMeshAddress]); }, [filteredMeshMessages, meshSessionActive, publicMeshAddress]);
useEffect(() => {
if (!expanded || activeTab !== 'meshtastic') return;
let alive = true;
const tick = async () => {
const data = await refreshMeshMqttSettings();
if (!alive || !data) return;
if (!data.enabled && meshSessionActive) {
setMeshQuickStatus({
type: 'err',
text: 'Public Mesh key is ready, but MQTT is off. Enable MQTT in Settings to join the live public lane.',
});
}
};
void tick();
const timer = window.setInterval(() => {
void tick();
}, meshMqttEnabled && !meshMqttConnected ? 5_000 : 15_000);
return () => {
alive = false;
window.clearInterval(timer);
};
}, [
activeTab,
expanded,
meshMqttConnected,
meshMqttEnabled,
meshSessionActive,
refreshMeshMqttSettings,
]);
// ─── InfoNet Polling ───────────────────────────────────────────────────── // ─── InfoNet Polling ─────────────────────────────────────────────────────
@@ -1735,7 +1961,7 @@ export function useMeshChatController({
// ─── Meshtastic Channel Discovery ────────────────────────────────────── // ─── Meshtastic Channel Discovery ──────────────────────────────────────
useEffect(() => { useEffect(() => {
if (!expanded || activeTab !== 'meshtastic') return; if (!expanded || activeTab !== 'meshtastic' || !meshSessionActive) return;
let cancelled = false; let cancelled = false;
const fetchChannels = async () => { const fetchChannels = async () => {
try { try {
@@ -1794,12 +2020,12 @@ export function useMeshChatController({
cancelled = true; cancelled = true;
clearInterval(iv); clearInterval(iv);
}; };
}, [expanded, activeTab, meshRegion]); }, [expanded, activeTab, meshRegion, meshSessionActive]);
// ─── Meshtastic Polling ────────────────────────────────────────────────── // ─── Meshtastic Polling ──────────────────────────────────────────────────
useEffect(() => { useEffect(() => {
if (!expanded || activeTab !== 'meshtastic') return; if (!expanded || activeTab !== 'meshtastic' || !meshSessionActive) return;
let cancelled = false; let cancelled = false;
const poll = async () => { const poll = async () => {
try { try {
@@ -1823,7 +2049,13 @@ export function useMeshChatController({
cancelled = true; cancelled = true;
clearInterval(iv); clearInterval(iv);
}; };
}, [expanded, activeTab, meshRegion, meshChannel, meshView]); }, [expanded, activeTab, meshRegion, meshChannel, meshView, meshSessionActive]);
useEffect(() => {
if (meshSessionActive) return;
setMeshMessages([]);
setMeshQuickStatus(null);
}, [meshSessionActive]);
// ─── DM Polling ────────────────────────────────────────────────────────── // ─── DM Polling ──────────────────────────────────────────────────────────
@@ -2305,7 +2537,8 @@ export function useMeshChatController({
const handleSend = async () => { const handleSend = async () => {
const msg = inputValue.trim(); const msg = inputValue.trim();
if (!msg || !hasId || busy) return; if (!msg || busy) return;
if (activeTab !== 'meshtastic' && !hasId) return;
const cooldownMs = activeTab === 'dms' ? 0 : 30_000; const cooldownMs = activeTab === 'dms' ? 0 : 30_000;
const now = Date.now(); const now = Date.now();
@@ -2392,20 +2625,34 @@ export function useMeshChatController({
]); ]);
setGateReplyContext(null); setGateReplyContext(null);
} else if (activeTab === 'meshtastic') { } else if (activeTab === 'meshtastic') {
if (!publicIdentity || !hasSovereignty()) { if (!meshSessionActive || !publicMeshAddress) {
setInputValue(msg); setInputValue(msg);
setLastSendTime(0); setLastSendTime(0);
setSendError('public mesh identity needed'); setSendError(meshSessionActive ? 'public mesh identity needed' : 'meshchat is off');
openIdentityWizard({ openIdentityWizard({
type: 'err', type: 'err',
text: 'Quick fix: create a public mesh identity below, then retry your send.', text: hasStoredPublicLaneIdentity
? 'Quick fix: turn MeshChat on below, then retry your send.'
: 'Quick fix: create a public mesh identity below, then retry your send.',
}); });
setTimeout(() => setSendError(''), 4000); setTimeout(() => setSendError(''), 4000);
setBusy(false); setBusy(false);
return; return;
} }
if (!meshMqttEnabled) {
setInputValue(msg);
setLastSendTime(0);
setSendError('mqtt is off');
setMeshQuickStatus({
type: 'err',
text: 'Public Mesh key is ready, but MQTT is off. Open Settings and enable the public broker.',
});
setMeshView('settings');
setTimeout(() => setSendError(''), 4000);
setBusy(false);
return;
}
const meshDestination = meshDirectTarget.trim() || 'broadcast'; const meshDestination = meshDirectTarget.trim() || 'broadcast';
const sequence = nextSequence();
const payload = { const payload = {
message: msg, message: msg,
destination: meshDestination, destination: meshDestination,
@@ -2423,8 +2670,7 @@ export function useMeshChatController({
setBusy(false); setBusy(false);
return; return;
} }
const signature = await signEvent('message', publicIdentity.nodeId, sequence, payload); const sendRes = await fetch(`${API_BASE}/api/mesh/meshtastic/send`, {
const sendRes = await fetch(`${API_BASE}/api/mesh/send`, {
method: 'POST', method: 'POST',
headers: { 'Content-Type': 'application/json' }, headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ body: JSON.stringify({
@@ -2434,14 +2680,8 @@ export function useMeshChatController({
priority: 'normal', priority: 'normal',
ephemeral: false, ephemeral: false,
transport_lock: 'meshtastic', transport_lock: 'meshtastic',
sender_id: publicIdentity.nodeId, sender_id: publicMeshAddress,
node_id: publicIdentity.nodeId, mesh_region: meshRegion,
public_key: publicIdentity.publicKey,
public_key_algo: getPublicKeyAlgo(),
signature,
sequence,
protocol_version: PROTOCOL_VERSION,
credentials: { mesh_region: meshRegion },
}), }),
}); });
if (!sendRes.ok) { if (!sendRes.ok) {
@@ -2455,15 +2695,7 @@ export function useMeshChatController({
if (!sendData.ok) { if (!sendData.ok) {
setInputValue(msg); setInputValue(msg);
setLastSendTime(0); setLastSendTime(0);
if (sendData.detail === 'Invalid signature') { setSendError(sendData.detail || 'send failed');
setSendError('public mesh signature failed');
openIdentityWizard({
type: 'err',
text: 'This public mesh identity did not verify. Reset it, recreate it, then retry.',
});
} else {
setSendError(sendData.detail || 'send failed');
}
setTimeout(() => setSendError(''), 4000); setTimeout(() => setSendError(''), 4000);
return; return;
} }
@@ -3915,7 +4147,8 @@ export function useMeshChatController({
wormholeEnabled && wormholeEnabled &&
wormholeReadyState && wormholeReadyState &&
!selectedGateAccessReady) || !selectedGateAccessReady) ||
((activeTab === 'infonet' || activeTab === 'meshtastic') && anonymousPublicBlocked) || (activeTab === 'infonet' && anonymousPublicBlocked) ||
(activeTab === 'meshtastic' && (!hasPublicLaneIdentity || !meshMqttEnabled)) ||
(activeTab === 'dms' && (activeTab === 'dms' &&
(dmView !== 'chat' || (dmView !== 'chat' ||
!selectedContact || !selectedContact ||
@@ -3959,16 +4192,36 @@ export function useMeshChatController({
[inputDisabled], [inputDisabled],
); );
const disableWormholeForPublicMesh = useCallback(async () => {
const requireBackendLeave = wormholeEnabled || wormholeReadyState;
try {
await leaveWormhole();
} catch (err) {
if (requireBackendLeave) {
throw err;
}
}
setWormholeEnabled(false);
setWormholeReadyState(false);
setWormholeRnsReady(false);
setWormholeRnsDirectReady(false);
setWormholeRnsPeers({ active: 0, configured: 0 });
setSecureModeCached(false);
}, [wormholeEnabled, wormholeReadyState]);
const createPublicMeshIdentity = useCallback( const createPublicMeshIdentity = useCallback(
async ({ closeWizardOnSuccess }: { closeWizardOnSuccess: boolean }) => { async ({ closeWizardOnSuccess }: { closeWizardOnSuccess: boolean }) => {
setIdentityWizardBusy(true); setIdentityWizardBusy(true);
setIdentityWizardStatus(null); setIdentityWizardStatus(null);
try { try {
const nextIdentity = await generateNodeKeys(); await disableWormholeForPublicMesh();
const nextAddress = await derivePublicMeshAddress(nextIdentity.nodeId).catch(() => ''); const nextAddress = createPublicMeshAddress();
const readyAddress = (nextAddress || nextIdentity.nodeId).toUpperCase(); await enableMeshMqttBridge();
setIdentity(nextIdentity); writeStoredPublicMeshAddress(nextAddress);
setPublicMeshAddress(nextAddress || nextIdentity.nodeId); const readyAddress = nextAddress.toUpperCase();
setPublicMeshAddress(nextAddress);
setMeshSessionActive(true);
setMeshMessages([]);
setSendError(''); setSendError('');
const successText = `Mesh key ready. Address ${readyAddress} is live for this testnet session.`; const successText = `Mesh key ready. Address ${readyAddress} is live for this testnet session.`;
setIdentityWizardStatus({ setIdentityWizardStatus({
@@ -3997,7 +4250,7 @@ export function useMeshChatController({
setIdentityWizardBusy(false); setIdentityWizardBusy(false);
} }
}, },
[], [disableWormholeForPublicMesh, enableMeshMqttBridge],
); );
const handleCreatePublicIdentity = useCallback(async () => { const handleCreatePublicIdentity = useCallback(async () => {
@@ -4013,6 +4266,44 @@ export function useMeshChatController({
} }
}, [createPublicMeshIdentity]); }, [createPublicMeshIdentity]);
const handleActivatePublicMeshSession = useCallback(async () => {
setIdentityWizardBusy(true);
setIdentityWizardStatus(null);
setMeshQuickStatus(null);
try {
const savedAddress = readStoredPublicMeshAddress();
if (!savedAddress) {
const text = 'No saved public mesh key is available. Create a mesh key first.';
setMeshSessionActive(false);
setIdentityWizardStatus({ type: 'err', text });
setMeshQuickStatus({ type: 'err', text });
return { ok: false as const, text };
}
await disableWormholeForPublicMesh();
await enableMeshMqttBridge();
const readyAddress = savedAddress.toUpperCase();
setPublicMeshAddress(savedAddress);
setMeshSessionActive(true);
setMeshMessages([]);
setSendError('');
const text = `MeshChat is on with saved address ${readyAddress}.`;
setIdentityWizardStatus({ type: 'ok', text });
setMeshQuickStatus({ type: 'ok', text });
return { ok: true as const, text };
} catch (err) {
const message =
typeof err === 'object' && err !== null && 'message' in err
? String((err as { message?: string }).message)
: 'unknown error';
const text = `Could not turn MeshChat on: ${message}`;
setIdentityWizardStatus({ type: 'err', text });
setMeshQuickStatus({ type: 'err', text });
return { ok: false as const, text };
} finally {
setIdentityWizardBusy(false);
}
}, [disableWormholeForPublicMesh, enableMeshMqttBridge]);
const handleReplyToMeshAddress = useCallback((address: string) => { const handleReplyToMeshAddress = useCallback((address: string) => {
const target = String(address || '').trim(); const target = String(address || '').trim();
if (!target) return; if (!target) return;
@@ -4023,36 +4314,16 @@ export function useMeshChatController({
}, []); }, []);
const handleLeaveWormholeForPublicMesh = useCallback(async () => { const handleLeaveWormholeForPublicMesh = useCallback(async () => {
setIdentityWizardBusy(true); const result = hasStoredPublicLaneIdentity
setIdentityWizardStatus(null); ? await handleActivatePublicMeshSession()
setMeshQuickStatus(null); : await createPublicMeshIdentity({ closeWizardOnSuccess: false });
try { const status = { type: result.ok ? 'ok' as const : 'err' as const, text: result.text };
await leaveWormhole(); setIdentityWizardStatus(status);
setWormholeEnabled(false); setMeshQuickStatus(status);
setWormholeReadyState(false); if (result.ok) {
setWormholeRnsReady(false); window.setTimeout(() => setIdentityWizardOpen(false), 900);
setWormholeRnsDirectReady(false);
setWormholeRnsPeers({ active: 0, configured: 0 });
setSecureModeCached(false);
const result = await createPublicMeshIdentity({ closeWizardOnSuccess: false });
const status = { type: result.ok ? 'ok' as const : 'err' as const, text: result.text };
setIdentityWizardStatus(status);
setMeshQuickStatus(status);
if (result.ok) {
window.setTimeout(() => setIdentityWizardOpen(false), 900);
}
} catch (err) {
const message =
typeof err === 'object' && err !== null && 'message' in err
? String((err as { message?: string }).message)
: 'unknown error';
const text = `Could not turn Wormhole off for public mesh: ${message}`;
setIdentityWizardStatus({ type: 'err', text });
setMeshQuickStatus({ type: 'err', text });
} finally {
setIdentityWizardBusy(false);
} }
}, [createPublicMeshIdentity]); }, [createPublicMeshIdentity, handleActivatePublicMeshSession, hasStoredPublicLaneIdentity]);
const handleResetPublicIdentity = useCallback(async () => { const handleResetPublicIdentity = useCallback(async () => {
if (wormholeEnabled && wormholeReadyState) { if (wormholeEnabled && wormholeReadyState) {
@@ -4065,13 +4336,10 @@ export function useMeshChatController({
setIdentityWizardBusy(true); setIdentityWizardBusy(true);
setIdentityWizardStatus(null); setIdentityWizardStatus(null);
try { try {
await clearBrowserIdentityState(); setMeshSessionActive(false);
setIdentity(null); setMeshMessages([]);
setContacts({}); clearStoredPublicMeshAddress();
setSelectedContact(''); setPublicMeshAddress('');
setDmMessages([]);
setAccessRequestsState([]);
setPendingSentState([]);
setIdentityWizardStatus({ setIdentityWizardStatus({
type: 'ok', type: 'ok',
text: 'Public mesh identity cleared. Start a fresh one when you are ready.', text: 'Public mesh identity cleared. Start a fresh one when you are ready.',
@@ -4091,6 +4359,8 @@ export function useMeshChatController({
}, [wormholeEnabled, wormholeReadyState]); }, [wormholeEnabled, wormholeReadyState]);
const handleBootstrapPrivateIdentity = useCallback(async () => { const handleBootstrapPrivateIdentity = useCallback(async () => {
setMeshSessionActive(false);
setMeshMessages([]);
if (wormholeEnabled && wormholeReadyState) { if (wormholeEnabled && wormholeReadyState) {
setIdentityWizardStatus({ setIdentityWizardStatus({
type: 'ok', type: 'ok',
@@ -4175,14 +4445,27 @@ export function useMeshChatController({
identityWizardStatus, identityWizardStatus,
setIdentityWizardStatus, setIdentityWizardStatus,
meshQuickStatus, meshQuickStatus,
meshSessionActive,
publicMeshAddress, publicMeshAddress,
meshView, meshView,
setMeshView, setMeshView,
meshDirectTarget, meshDirectTarget,
setMeshDirectTarget, setMeshDirectTarget,
meshMqttSettings,
meshMqttForm,
setMeshMqttForm,
meshMqttBusy,
meshMqttStatusText,
meshMqttEnabled,
meshMqttRunning,
meshMqttConnected,
meshMqttConnectionLabel,
saveMeshMqttSettings,
refreshMeshMqttSettings,
// Identity // Identity
identity, identity,
publicIdentity, publicIdentity,
hasStoredPublicLaneIdentity,
hasPublicLaneIdentity, hasPublicLaneIdentity,
hasId, hasId,
shouldShowIdentityWarning, shouldShowIdentityWarning,
@@ -4320,6 +4603,7 @@ export function useMeshChatController({
openChat, openChat,
handleCreatePublicIdentity, handleCreatePublicIdentity,
handleQuickCreatePublicIdentity, handleQuickCreatePublicIdentity,
handleActivatePublicMeshSession,
handleLeaveWormholeForPublicMesh, handleLeaveWormholeForPublicMesh,
handleResetPublicIdentity, handleResetPublicIdentity,
handleBootstrapPrivateIdentity, handleBootstrapPrivateIdentity,
+11 -3
View File
@@ -364,6 +364,7 @@ function summarizeNodePeer(peerUrl?: string): string {
} }
function describeBootstrapState(snapshot?: InfonetNodeStatusSnapshot | null): string { function describeBootstrapState(snapshot?: InfonetNodeStatusSnapshot | null): string {
if (snapshot && !snapshot.node_enabled) return 'READY / DISABLED';
const bootstrap = snapshot?.bootstrap; const bootstrap = snapshot?.bootstrap;
if (!bootstrap) return 'LOCAL ONLY'; if (!bootstrap) return 'LOCAL ONLY';
if (bootstrap.manifest_loaded) { if (bootstrap.manifest_loaded) {
@@ -376,6 +377,7 @@ function describeBootstrapState(snapshot?: InfonetNodeStatusSnapshot | null): st
} }
function describeSyncOutcome(snapshot?: InfonetNodeStatusSnapshot | null): string { function describeSyncOutcome(snapshot?: InfonetNodeStatusSnapshot | null): string {
if (snapshot && !snapshot.node_enabled) return 'OFF - click NODE to activate';
const sync = snapshot?.sync_runtime; const sync = snapshot?.sync_runtime;
if (!sync) return 'IDLE'; if (!sync) return 'IDLE';
const outcome = String(sync.last_outcome || 'idle').trim().toLowerCase(); const outcome = String(sync.last_outcome || 'idle').trim().toLowerCase();
@@ -433,6 +435,12 @@ function buildNodeRuntimeLines(snapshot: InfonetNodeStatusSnapshot): TermLine[]
type: 'error', type: 'error',
}); });
} }
if (!snapshot.node_enabled) {
lines.push({
text: ' Activate: click the NODE button in the top-right controls to join the public testnet seed',
type: 'dim',
});
}
lines.push({ text: '', type: 'dim' }); lines.push({ text: '', type: 'dim' });
return lines; return lines;
} }
@@ -5945,7 +5953,7 @@ export default function MeshTerminal({ isOpen, launchToken = 0, onClose, onDmCou
PARTICIPANT NODE PARTICIPANT NODE
</div> </div>
<div className="mt-1 text-sm leading-5 text-slate-400"> <div className="mt-1 text-sm leading-5 text-slate-400">
Automatic bootstrap and sync now live on the backend lane. This node can keep a local chain even with Wormhole off. Backend bootstrap is configured; the participant node syncs the testnet seed over the private seed lane.
</div> </div>
</div> </div>
<div className="border border-cyan-500/20 bg-cyan-500/8 px-3 py-1.5 text-[13px] tracking-[0.22em] text-cyan-200"> <div className="border border-cyan-500/20 bg-cyan-500/8 px-3 py-1.5 text-[13px] tracking-[0.22em] text-cyan-200">
@@ -6000,10 +6008,10 @@ export default function MeshTerminal({ isOpen, launchToken = 0, onClose, onDmCou
<div className="border border-amber-400/16 bg-amber-400/6 px-4 py-3 text-sm leading-6 text-amber-100/85"> <div className="border border-amber-400/16 bg-amber-400/6 px-4 py-3 text-sm leading-6 text-amber-100/85">
<div className="text-[13px] font-mono tracking-[0.24em] text-amber-300"> <div className="text-[13px] font-mono tracking-[0.24em] text-amber-300">
WORMHOLE OPTIONAL FOR NODE SYNC PRIVATE SEED LANE
</div> </div>
<div className="mt-2"> <div className="mt-2">
Participant-node bootstrap, sync, and public chain hosting run on the backend lane without Wormhole. Participant-node bootstrap, sync, and public chain hosting use the backend private seed lane.
</div> </div>
<div className="mt-2 text-amber-200/75"> <div className="mt-2 text-amber-200/75">
Turn Wormhole on for gates, obfuscated inbox, and the stronger obfuscated lane only. Turn Wormhole on for gates, obfuscated inbox, and the stronger obfuscated lane only.
+266 -37
View File
@@ -7,6 +7,7 @@ import React, { useEffect, useRef, useCallback } from 'react';
import WikiImage from '@/components/WikiImage'; import WikiImage from '@/components/WikiImage';
import type { SelectedEntity, RegionDossier, FimiData } from "@/types/dashboard"; import type { SelectedEntity, RegionDossier, FimiData } from "@/types/dashboard";
import { useDataKeys } from '@/hooks/useDataStore'; import { useDataKeys } from '@/hooks/useDataStore';
import { API_BASE } from '@/lib/api';
import { lookupShodanHost } from '@/lib/shodanClient'; import { lookupShodanHost } from '@/lib/shodanClient';
import type { ShodanHost } from '@/types/shodan'; import type { ShodanHost } from '@/types/shodan';
@@ -100,6 +101,7 @@ const AIRCRAFT_WIKI: Record<string, string> = {
PA46: 'Piper PA-46 Malibu', BE36: 'Beechcraft Bonanza', BE9L: 'Beechcraft King Air', PA46: 'Piper PA-46 Malibu', BE36: 'Beechcraft Bonanza', BE9L: 'Beechcraft King Air',
BE20: 'Beechcraft Super King Air', B350: 'Beechcraft King Air 350', PC12: 'Pilatus PC-12', BE20: 'Beechcraft Super King Air', B350: 'Beechcraft King Air 350', PC12: 'Pilatus PC-12',
PC24: 'Pilatus PC-24', TBM7: 'Daher TBM', TBM8: 'Daher TBM', TBM9: 'Daher TBM', PC24: 'Pilatus PC-24', TBM7: 'Daher TBM', TBM8: 'Daher TBM', TBM9: 'Daher TBM',
PIVI: 'Pipistrel Virus',
// Helicopters // Helicopters
R44: 'Robinson R44', R22: 'Robinson R22', R66: 'Robinson R66', R44: 'Robinson R44', R22: 'Robinson R22', R66: 'Robinson R66',
B06: 'Bell 206', B407: 'Bell 407', B412: 'Bell 412', B06: 'Bell 206', B407: 'Bell 407', B412: 'Bell 412',
@@ -196,12 +198,17 @@ function resolveAcTypeWiki(acType: string): string | null {
return null; return null;
} }
function resolveAircraftWikiTitle(model: string | undefined): string | null {
if (!model) return null;
return AIRCRAFT_WIKI[model] || resolveAcTypeWiki(model);
}
// Module-level cache for Wikipedia thumbnails (persists across re-renders) // Module-level cache for Wikipedia thumbnails (persists across re-renders)
const _wikiThumbCache: Record<string, { url: string | null; loading: boolean }> = {}; const _wikiThumbCache: Record<string, { url: string | null; loading: boolean }> = {};
function useAircraftImage(model: string | undefined): { imgUrl: string | null; wikiUrl: string | null; loading: boolean } { function useAircraftImage(model: string | undefined): { imgUrl: string | null; wikiUrl: string | null; loading: boolean } {
const [, forceUpdate] = useState(0); const [, forceUpdate] = useState(0);
const wikiTitle = model ? AIRCRAFT_WIKI[model] : undefined; const wikiTitle = resolveAircraftWikiTitle(model) || undefined;
const wikiUrl = wikiTitle ? `https://en.wikipedia.org/wiki/${wikiTitle.replace(/ /g, '_')}` : null; const wikiUrl = wikiTitle ? `https://en.wikipedia.org/wiki/${wikiTitle.replace(/ /g, '_')}` : null;
useEffect(() => { useEffect(() => {
@@ -236,6 +243,164 @@ const VESSEL_TYPE_WIKI: Record<string, string> = {
'military_vessel': 'https://en.wikipedia.org/wiki/Warship', 'military_vessel': 'https://en.wikipedia.org/wiki/Warship',
}; };
type FlightTrailPoint = { lat?: number; lng?: number; alt?: number; ts?: number } | number[];
function readTrailTimestamp(point: FlightTrailPoint): number | null {
if (Array.isArray(point)) {
const ts = Number(point[3]);
return Number.isFinite(ts) && ts > 0 ? ts : null;
}
const ts = Number(point?.ts);
return Number.isFinite(ts) && ts > 0 ? ts : null;
}
function readTrailLatLng(point: FlightTrailPoint): { lat: number; lng: number } | null {
const lat = Number(Array.isArray(point) ? point[0] : point?.lat);
const lng = Number(Array.isArray(point) ? point[1] : point?.lng);
if (!Number.isFinite(lat) || !Number.isFinite(lng)) return null;
return { lat, lng };
}
function distanceNm(a: { lat: number; lng: number }, b: { lat: number; lng: number }): number {
const toRad = (deg: number) => (deg * Math.PI) / 180;
const earthRadiusNm = 3440.065;
const dLat = toRad(b.lat - a.lat);
const dLng = toRad(b.lng - a.lng);
const lat1 = toRad(a.lat);
const lat2 = toRad(b.lat);
const h =
Math.sin(dLat / 2) ** 2 +
Math.cos(lat1) * Math.cos(lat2) * Math.sin(dLng / 2) ** 2;
return 2 * earthRadiusNm * Math.atan2(Math.sqrt(h), Math.sqrt(1 - h));
}
function formatObservedDuration(hours: number): string {
const minutes = Math.max(1, Math.round(hours * 60));
if (minutes < 60) return `${minutes} min`;
const wholeHours = Math.floor(minutes / 60);
const remainder = minutes % 60;
return remainder ? `${wholeHours}h ${remainder}m` : `${wholeHours}h`;
}
function estimateObservedEmissions(flight: any): {
fuelGallons: number;
co2Kg: number;
durationLabel: string;
distanceLabel: string | null;
basisLabel: string;
} | null {
const fuelGph = Number(flight?.emissions?.fuel_gph);
const co2KgPerHour = Number(flight?.emissions?.co2_kg_per_hour);
const trail = Array.isArray(flight?.trail) ? (flight.trail as FlightTrailPoint[]) : [];
const isTrackedAircraft = flight?.type === 'tracked_flight' || Boolean(flight?.alert_category);
const minimumObservedHours = isTrackedAircraft ? 1 / 60 : 5 / 60;
if (!Number.isFinite(fuelGph) || !Number.isFinite(co2KgPerHour)) {
return null;
}
const timestamps = trail
.map(readTrailTimestamp)
.filter((ts): ts is number => ts !== null)
.sort((a, b) => a - b);
if (timestamps.length >= 2) {
const elapsedHours = (timestamps[timestamps.length - 1] - timestamps[0]) / 3600;
if (Number.isFinite(elapsedHours) && elapsedHours >= minimumObservedHours) {
let distance = 0;
let previous: { lat: number; lng: number } | null = null;
for (const point of trail) {
const current = readTrailLatLng(point);
if (previous && current) distance += distanceNm(previous, current);
if (current) previous = current;
}
return {
fuelGallons: Math.round(fuelGph * elapsedHours),
co2Kg: Math.round(co2KgPerHour * elapsedHours),
durationLabel: formatObservedDuration(elapsedHours),
distanceLabel: distance > 1 ? `${Math.round(distance).toLocaleString()} nm` : null,
basisLabel: 'trail history',
};
}
}
const origin = Array.isArray(flight?.origin_loc)
? { lng: Number(flight.origin_loc[0]), lat: Number(flight.origin_loc[1]) }
: null;
const current = { lat: Number(flight?.lat), lng: Number(flight?.lng) };
const speedKnots = Number(flight?.speed_knots);
if (
origin &&
Number.isFinite(origin.lat) &&
Number.isFinite(origin.lng) &&
Number.isFinite(current.lat) &&
Number.isFinite(current.lng) &&
Number.isFinite(speedKnots) &&
speedKnots > 50
) {
const flownNm = distanceNm(origin, current);
const elapsedHours = flownNm / speedKnots;
if (Number.isFinite(elapsedHours) && elapsedHours >= minimumObservedHours && elapsedHours <= 18) {
return {
fuelGallons: Math.round(fuelGph * elapsedHours),
co2Kg: Math.round(co2KgPerHour * elapsedHours),
durationLabel: formatObservedDuration(elapsedHours),
distanceLabel: `${Math.round(flownNm).toLocaleString()} nm`,
basisLabel: 'route progress',
};
}
}
return null;
}
function EmissionsEstimateBlock({ flight }: { flight: any }) {
const observed = estimateObservedEmissions(flight);
const emissions = flight?.emissions;
const context = observed
? `${observed.durationLabel} ${observed.basisLabel}${observed.distanceLabel ? ` / ${observed.distanceLabel}` : ''}`
: emissions
? 'Rate only until enough trail history accumulates'
: null;
return (
<div className="border-b border-[var(--border-primary)] pb-2">
<span className="text-[var(--text-muted)] text-[10px] block mb-1.5">EMISSIONS ESTIMATE</span>
<div className="flex gap-3">
<div className="flex-1 bg-[var(--bg-primary)]/50 border border-[var(--border-primary)] px-2 py-1.5">
<div className="text-[11px] text-[var(--text-muted)] tracking-widest">
{observed ? 'FUEL BURNED' : 'FUEL RATE'}
</div>
<div className="text-xs font-bold text-orange-400">
{observed ? (
<>{observed.fuelGallons.toLocaleString()} <span className="text-[11px] text-[var(--text-muted)] font-normal">GAL</span></>
) : emissions ? (
<>{emissions.fuel_gph} <span className="text-[11px] text-[var(--text-muted)] font-normal">GPH</span></>
) : 'UNKNOWN'}
</div>
</div>
<div className="flex-1 bg-[var(--bg-primary)]/50 border border-[var(--border-primary)] px-2 py-1.5">
<div className="text-[11px] text-[var(--text-muted)] tracking-widest">
{observed ? 'CO2 PRODUCED' : 'CO2 RATE'}
</div>
<div className="text-xs font-bold text-red-400">
{observed ? (
<>{observed.co2Kg.toLocaleString()} <span className="text-[11px] text-[var(--text-muted)] font-normal">KG</span></>
) : emissions ? (
<>{emissions.co2_kg_per_hour.toLocaleString()} <span className="text-[11px] text-[var(--text-muted)] font-normal">KG/HR</span></>
) : 'UNKNOWN'}
</div>
</div>
</div>
{context && (
<div className="mt-1.5 text-[10px] text-[var(--text-muted)] leading-relaxed">
{context}
{observed && emissions ? ` - estimated from ${emissions.fuel_gph} GPH model rate.` : ''}
</div>
)}
</div>
);
}
function NewsFeedInner({ selectedEntity, regionDossier, regionDossierLoading, onArticleClick }: { selectedEntity?: SelectedEntity | null, regionDossier?: RegionDossier | null, regionDossierLoading?: boolean, onArticleClick?: (idx: number, lat?: number, lng?: number, title?: string) => void }) { function NewsFeedInner({ selectedEntity, regionDossier, regionDossierLoading, onArticleClick }: { selectedEntity?: SelectedEntity | null, regionDossier?: RegionDossier | null, regionDossierLoading?: boolean, onArticleClick?: (idx: number, lat?: number, lng?: number, title?: string) => void }) {
const data = useDataKeys([ const data = useDataKeys([
'news', 'fimi', 'commercial_flights', 'private_flights', 'private_jets', 'news', 'fimi', 'commercial_flights', 'private_flights', 'private_jets',
@@ -243,6 +408,7 @@ function NewsFeedInner({ selectedEntity, regionDossier, regionDossierLoading, on
'airports', 'last_updated', 'threat_level', 'airports', 'last_updated', 'threat_level',
] as const); ] as const);
const [isMinimized, setIsMinimized] = useState(false); const [isMinimized, setIsMinimized] = useState(false);
const [selectedFlightTrail, setSelectedFlightTrail] = useState<FlightTrailPoint[]>([]);
const [expandedIndexes, setExpandedIndexes] = useState<number[]>([]); const [expandedIndexes, setExpandedIndexes] = useState<number[]>([]);
const [fimiExpanded, setFimiExpanded] = useState(false); const [fimiExpanded, setFimiExpanded] = useState(false);
const [aiSummaryOpen, setAiSummaryOpen] = useState(false); const [aiSummaryOpen, setAiSummaryOpen] = useState(false);
@@ -277,15 +443,72 @@ function NewsFeedInner({ selectedEntity, regionDossier, regionDossierLoading, on
const selectedFlightModel = (() => { const selectedFlightModel = (() => {
if (!selectedEntity) return undefined; if (!selectedEntity) return undefined;
const { type, id } = selectedEntity; const { type, id } = selectedEntity;
let flight: any = null; const findByIdOrIndex = (flights?: Array<{ icao24?: string; model?: string }>) => {
if (type === 'flight') flight = data?.commercial_flights?.[id as number]; if (!flights) return null;
else if (type === 'private_flight') flight = data?.private_flights?.[id as number]; if (typeof id === 'number') return flights[id] || null;
else if (type === 'private_jet') flight = data?.private_jets?.[id as number]; return flights.find((flight) => flight.icao24 === id) || null;
else if (type === 'military_flight') flight = data?.military_flights?.[id as number]; };
else if (type === 'tracked_flight') flight = data?.tracked_flights?.[id as number]; let flight: { model?: string } | null = null;
if (type === 'flight') flight = findByIdOrIndex(data?.commercial_flights);
else if (type === 'private_flight') flight = findByIdOrIndex(data?.private_flights);
else if (type === 'private_jet') flight = findByIdOrIndex(data?.private_jets);
else if (type === 'military_flight') flight = findByIdOrIndex(data?.military_flights);
else if (type === 'tracked_flight') flight = findByIdOrIndex(data?.tracked_flights);
return flight?.model; return flight?.model;
})(); })();
const { imgUrl: aircraftImgUrl, wikiUrl: aircraftWikiUrl, loading: aircraftImgLoading } = useAircraftImage(selectedFlightModel); const { imgUrl: aircraftImgUrl, wikiUrl: aircraftWikiUrl, loading: aircraftImgLoading } = useAircraftImage(selectedFlightModel);
useEffect(() => {
const flightSelectionTypes = new Set([
'flight',
'commercial_flight',
'private_flight',
'private_ga',
'private_jet',
'military_flight',
'tracked_flight',
]);
if (!selectedEntity || !flightSelectionTypes.has(selectedEntity.type)) {
setSelectedFlightTrail([]);
return;
}
const trailId = String(selectedEntity.id || '').trim();
if (!trailId) {
setSelectedFlightTrail([]);
return;
}
let cancelled = false;
const refreshSelectedFlightTrail = () => {
fetch(`${API_BASE}/api/trail/flight/${encodeURIComponent(trailId)}`, { cache: 'no-store' })
.then((res) => (res.ok ? res.json() : null))
.then((payload) => {
if (cancelled) return;
const trail = Array.isArray(payload?.trail) ? payload.trail as FlightTrailPoint[] : [];
setSelectedFlightTrail(trail);
})
.catch(() => {
if (!cancelled) setSelectedFlightTrail([]);
});
};
refreshSelectedFlightTrail();
const trailRefreshTimer = window.setInterval(refreshSelectedFlightTrail, 30000);
return () => {
cancelled = true;
window.clearInterval(trailRefreshTimer);
};
}, [selectedEntity?.id, selectedEntity?.type]);
const withSelectedTrail = useCallback((flight: any) => {
if (!flight || selectedFlightTrail.length < 2) return flight;
const selectedId = String(selectedEntity?.id || '').trim().toLowerCase();
const flightId = String(flight.icao24 || '').trim().toLowerCase();
if (!selectedId || !flightId || selectedId !== flightId) return flight;
return { ...flight, trail: selectedFlightTrail };
}, [selectedEntity?.id, selectedFlightTrail]);
const [shodanDetail, setShodanDetail] = useState<ShodanHost | null>(null); const [shodanDetail, setShodanDetail] = useState<ShodanHost | null>(null);
const [shodanLoading, setShodanLoading] = useState(false); const [shodanLoading, setShodanLoading] = useState(false);
const [shodanError, setShodanError] = useState<string | null>(null); const [shodanError, setShodanError] = useState<string | null>(null);
@@ -499,6 +722,7 @@ function NewsFeedInner({ selectedEntity, regionDossier, regionDossierLoading, on
if (selectedEntity?.type === 'tracked_flight') { if (selectedEntity?.type === 'tracked_flight') {
const flight = data?.tracked_flights?.find((f: any) => f.icao24 === selectedEntity.id); const flight = data?.tracked_flights?.find((f: any) => f.icao24 === selectedEntity.id);
if (flight) { if (flight) {
const flightForEmissions = withSelectedTrail(flight);
const callsign = flight.callsign || "UNKNOWN"; const callsign = flight.callsign || "UNKNOWN";
const alertColorMap: Record<string, string> = { const alertColorMap: Record<string, string> = {
'#ff1493': 'text-[#ff1493]', pink: 'text-[#ff1493]', red: 'text-red-400', yellow: 'text-yellow-400', '#ff1493': 'text-[#ff1493]', pink: 'text-[#ff1493]', red: 'text-red-400', yellow: 'text-yellow-400',
@@ -684,19 +908,7 @@ function NewsFeedInner({ selectedEntity, regionDossier, regionDossierLoading, on
<span className={`text-xs font-bold ${flight.squawk === '7700' ? 'text-red-400 animate-pulse' : flight.squawk === '7600' ? 'text-yellow-400' : 'text-[var(--text-primary)]'}`}>{flight.squawk}{flight.squawk === '7700' ? ' ⚠ EMERGENCY' : flight.squawk === '7600' ? ' COMMS LOST' : ''}</span> <span className={`text-xs font-bold ${flight.squawk === '7700' ? 'text-red-400 animate-pulse' : flight.squawk === '7600' ? 'text-yellow-400' : 'text-[var(--text-primary)]'}`}>{flight.squawk}{flight.squawk === '7700' ? ' ⚠ EMERGENCY' : flight.squawk === '7600' ? ' COMMS LOST' : ''}</span>
</div> </div>
)} )}
<div className="border-b border-[var(--border-primary)] pb-2"> <EmissionsEstimateBlock flight={flightForEmissions} />
<span className="text-[var(--text-muted)] text-[10px] block mb-1.5">EMISSIONS ESTIMATE</span>
<div className="flex gap-3">
<div className="flex-1 bg-[var(--bg-primary)]/50 border border-[var(--border-primary)] px-2 py-1.5">
<div className="text-[11px] text-[var(--text-muted)] tracking-widest">FUEL BURN</div>
<div className="text-xs font-bold text-orange-400">{flight.emissions ? <>{flight.emissions.fuel_gph} <span className="text-[11px] text-[var(--text-muted)] font-normal">GPH</span></> : 'UNKNOWN'}</div>
</div>
<div className="flex-1 bg-[var(--bg-primary)]/50 border border-[var(--border-primary)] px-2 py-1.5">
<div className="text-[11px] text-[var(--text-muted)] tracking-widest">CO2 OUTPUT</div>
<div className="text-xs font-bold text-red-400">{flight.emissions ? <>{flight.emissions.co2_kg_per_hour.toLocaleString()} <span className="text-[11px] text-[var(--text-muted)] font-normal">KG/HR</span></> : 'UNKNOWN'}</div>
</div>
</div>
</div>
{flight.alert_link && ( {flight.alert_link && (
<div className="flex justify-between items-center border-b border-[var(--border-primary)] pb-2"> <div className="flex justify-between items-center border-b border-[var(--border-primary)] pb-2">
<span className="text-[var(--text-muted)] text-[10px]">REFERENCE</span> <span className="text-[var(--text-muted)] text-[10px]">REFERENCE</span>
@@ -748,8 +960,15 @@ function NewsFeedInner({ selectedEntity, regionDossier, regionDossierLoading, on
const flight = flightsList?.find((f: any) => f.icao24 === selectedEntity.id); const flight = flightsList?.find((f: any) => f.icao24 === selectedEntity.id);
if (flight) { if (flight) {
const flightForEmissions = withSelectedTrail(flight);
const callsign = flight.callsign || "UNKNOWN"; const callsign = flight.callsign || "UNKNOWN";
let airline = "UNKNOWN"; let airline = "UNKNOWN";
const isPrivateFlight = selectedEntity.type === 'private_flight' || selectedEntity.type === 'private_jet';
const aircraftWikiTitle = resolveAircraftWikiTitle(flight.model);
const aircraftModelWikiUrl = aircraftWikiTitle
? `https://en.wikipedia.org/wiki/${aircraftWikiTitle.replace(/ /g, '_')}`
: null;
const showModelWiki = isPrivateFlight || selectedEntity.type === 'military_flight';
if (selectedEntity.type === 'military_flight') { if (selectedEntity.type === 'military_flight') {
const mil = flight as import('@/types/dashboard').MilitaryFlight; const mil = flight as import('@/types/dashboard').MilitaryFlight;
@@ -798,7 +1017,7 @@ function NewsFeedInner({ selectedEntity, regionDossier, regionDossierLoading, on
<div className="p-4 flex flex-col gap-3"> <div className="p-4 flex flex-col gap-3">
<div className="flex justify-between items-center border-b border-[var(--border-primary)] pb-2"> <div className="flex justify-between items-center border-b border-[var(--border-primary)] pb-2">
<span className="text-[var(--text-muted)] text-[10px]">OPERATOR</span> <span className="text-[var(--text-muted)] text-[10px]">OPERATOR</span>
{selectedEntity.type !== 'military_flight' && airline && airline !== 'COMMERCIAL FLIGHT' && airline !== 'UNKNOWN' ? ( {!isPrivateFlight && selectedEntity.type !== 'military_flight' && airline && airline !== 'COMMERCIAL FLIGHT' && airline !== 'UNKNOWN' ? (
<a <a
href={`https://en.wikipedia.org/wiki/${encodeURIComponent(airline.replace(/ /g, '_'))}`} href={`https://en.wikipedia.org/wiki/${encodeURIComponent(airline.replace(/ /g, '_'))}`}
target="_blank" target="_blank"
@@ -812,7 +1031,7 @@ function NewsFeedInner({ selectedEntity, regionDossier, regionDossierLoading, on
)} )}
</div> </div>
{/* Commercial: Airline company Wikipedia image */} {/* Commercial: Airline company Wikipedia image */}
{selectedEntity.type !== 'military_flight' && airline && airline !== 'COMMERCIAL FLIGHT' && airline !== 'UNKNOWN' && ( {!isPrivateFlight && selectedEntity.type !== 'military_flight' && airline && airline !== 'COMMERCIAL FLIGHT' && airline !== 'UNKNOWN' && (
<div className="border-b border-[var(--border-primary)] pb-2"> <div className="border-b border-[var(--border-primary)] pb-2">
<WikiImage <WikiImage
wikiUrl={`https://en.wikipedia.org/wiki/${encodeURIComponent(airline.replace(/ /g, '_'))}`} wikiUrl={`https://en.wikipedia.org/wiki/${encodeURIComponent(airline.replace(/ /g, '_'))}`}
@@ -828,7 +1047,18 @@ function NewsFeedInner({ selectedEntity, regionDossier, regionDossierLoading, on
</div> </div>
<div className="flex justify-between items-center border-b border-[var(--border-primary)] pb-2"> <div className="flex justify-between items-center border-b border-[var(--border-primary)] pb-2">
<span className="text-[var(--text-muted)] text-[10px]">AIRCRAFT MODEL</span> <span className="text-[var(--text-muted)] text-[10px]">AIRCRAFT MODEL</span>
<span className="text-[var(--text-primary)] text-xs font-bold">{flight.model || "UNKNOWN"}</span> {showModelWiki && aircraftModelWikiUrl ? (
<a
href={aircraftModelWikiUrl}
target="_blank"
rel="noreferrer"
className="text-xs font-bold text-cyan-400 hover:text-cyan-300 underline"
>
{aircraftWikiTitle || flight.model}
</a>
) : (
<span className="text-[var(--text-primary)] text-xs font-bold">{flight.model || "UNKNOWN"}</span>
)}
</div> </div>
{/* Military: Aircraft model Wikipedia image (gold accent) */} {/* Military: Aircraft model Wikipedia image (gold accent) */}
{selectedEntity.type === 'military_flight' && (() => { {selectedEntity.type === 'military_flight' && (() => {
@@ -878,8 +1108,19 @@ function NewsFeedInner({ selectedEntity, regionDossier, regionDossierLoading, on
} }
return null; return null;
})()} })()}
{/* Private/GA: aircraft model Wikipedia image as the primary visual */}
{isPrivateFlight && aircraftModelWikiUrl && (
<div className="border-b border-[var(--border-primary)] pb-3">
<WikiImage
wikiUrl={aircraftModelWikiUrl}
label={aircraftWikiTitle || flight.model}
maxH="max-h-36"
accent="hover:border-purple-400/60"
/>
</div>
)}
{/* Non-military: Aircraft model photo (secondary, below airline image) */} {/* Non-military: Aircraft model photo (secondary, below airline image) */}
{selectedEntity.type !== 'military_flight' && (aircraftImgUrl || aircraftImgLoading || aircraftWikiUrl) && ( {!isPrivateFlight && selectedEntity.type !== 'military_flight' && selectedEntity.type !== 'flight' && (aircraftImgUrl || aircraftImgLoading || aircraftWikiUrl) && (
<div className="border-b border-[var(--border-primary)] pb-3"> <div className="border-b border-[var(--border-primary)] pb-3">
{aircraftImgLoading && ( {aircraftImgLoading && (
<div className="w-full h-24 bg-[var(--bg-tertiary)]/60" /> <div className="w-full h-24 bg-[var(--bg-tertiary)]/60" />
@@ -924,19 +1165,7 @@ function NewsFeedInner({ selectedEntity, regionDossier, regionDossierLoading, on
<span className="text-[var(--text-muted)] text-[10px]">ROUTE</span> <span className="text-[var(--text-muted)] text-[10px]">ROUTE</span>
<span className="text-cyan-400 text-xs font-bold">{flight.origin_name !== "UNKNOWN" ? `[${flight.origin_name}] → [${flight.dest_name}]` : "UNKNOWN"}</span> <span className="text-cyan-400 text-xs font-bold">{flight.origin_name !== "UNKNOWN" ? `[${flight.origin_name}] → [${flight.dest_name}]` : "UNKNOWN"}</span>
</div> </div>
<div className="border-b border-[var(--border-primary)] pb-2"> <EmissionsEstimateBlock flight={flightForEmissions} />
<span className="text-[var(--text-muted)] text-[10px] block mb-1.5">EMISSIONS ESTIMATE</span>
<div className="flex gap-3">
<div className="flex-1 bg-[var(--bg-primary)]/50 border border-[var(--border-primary)] px-2 py-1.5">
<div className="text-[11px] text-[var(--text-muted)] tracking-widest">FUEL BURN</div>
<div className="text-xs font-bold text-orange-400">{flight.emissions ? <>{flight.emissions.fuel_gph} <span className="text-[11px] text-[var(--text-muted)] font-normal">GPH</span></> : 'UNKNOWN'}</div>
</div>
<div className="flex-1 bg-[var(--bg-primary)]/50 border border-[var(--border-primary)] px-2 py-1.5">
<div className="text-[11px] text-[var(--text-muted)] tracking-widest">CO2 OUTPUT</div>
<div className="text-xs font-bold text-red-400">{flight.emissions ? <>{flight.emissions.co2_kg_per_hour.toLocaleString()} <span className="text-[11px] text-[var(--text-muted)] font-normal">KG/HR</span></> : 'UNKNOWN'}</div>
</div>
</div>
</div>
{flight.icao24 && ( {flight.icao24 && (
<div className="flex justify-between items-center border-b border-[var(--border-primary)] pb-2"> <div className="flex justify-between items-center border-b border-[var(--border-primary)] pb-2">
<span className="text-[var(--text-muted)] text-[10px]">FLIGHT RECORD</span> <span className="text-[var(--text-muted)] text-[10px]">FLIGHT RECORD</span>
+385 -30
View File
@@ -2,9 +2,11 @@
import React, { useState, useEffect } from 'react'; import React, { useState, useEffect } from 'react';
import { motion, AnimatePresence } from 'framer-motion'; import { motion, AnimatePresence } from 'framer-motion';
import { X, ExternalLink, Key, Shield, Radar, Globe, Satellite, Ship, Radio } from 'lucide-react'; import { X, ExternalLink, Key, Shield, Radar, Globe, Satellite, Ship, Radio, Bot, Copy, Check, Network } from 'lucide-react';
const STORAGE_KEY = 'shadowbroker_onboarding_complete'; const CURRENT_ONBOARDING_VERSION = '0.9.75-agentic-onboarding-1';
const STORAGE_KEY = `shadowbroker_onboarding_complete_v${CURRENT_ONBOARDING_VERSION}`;
const LEGACY_STORAGE_KEY = 'shadowbroker_onboarding_complete';
const API_GUIDES = [ const API_GUIDES = [
{ {
@@ -17,7 +19,7 @@ const API_GUIDES = [
'Create a free account at opensky-network.org', 'Create a free account at opensky-network.org',
'Go to Dashboard → OAuth → Create Client', 'Go to Dashboard → OAuth → Create Client',
'Copy your Client ID and Client Secret', 'Copy your Client ID and Client Secret',
'Paste both into Settings → Aviation', 'Paste both into Quick Local Setup above or Settings → API Keys',
], ],
url: 'https://opensky-network.org/index.php?option=com_users&view=registration', url: 'https://opensky-network.org/index.php?option=com_users&view=registration',
color: 'cyan', color: 'cyan',
@@ -31,7 +33,7 @@ const API_GUIDES = [
'Register at aisstream.io', 'Register at aisstream.io',
'Navigate to your API Keys page', 'Navigate to your API Keys page',
'Generate a new API key', 'Generate a new API key',
'Paste it into Settings → Maritime', 'Paste it into Quick Local Setup above or Settings → API Keys',
], ],
url: 'https://aisstream.io/authenticate', url: 'https://aisstream.io/authenticate',
color: 'blue', color: 'blue',
@@ -59,18 +61,171 @@ const OnboardingModal = React.memo(function OnboardingModal({
onOpenSettings, onOpenSettings,
}: OnboardingModalProps) { }: OnboardingModalProps) {
const [step, setStep] = useState(0); const [step, setStep] = useState(0);
const [setupKeys, setSetupKeys] = useState({
OPENSKY_CLIENT_ID: '',
OPENSKY_CLIENT_SECRET: '',
AIS_API_KEY: '',
});
const [setupSaving, setSetupSaving] = useState(false);
const [setupMsg, setSetupMsg] = useState<{ type: 'ok' | 'err'; text: string } | null>(null);
const [agentSecret, setAgentSecret] = useState('');
const [agentTier, setAgentTier] = useState<'restricted' | 'full'>('restricted');
const [agentMode, setAgentMode] = useState<'local' | 'remote'>('local');
const [agentLoading, setAgentLoading] = useState(false);
const [agentMsg, setAgentMsg] = useState<{ type: 'ok' | 'err'; text: string } | null>(null);
const [agentCopied, setAgentCopied] = useState(false);
const [torStarting, setTorStarting] = useState(false);
const [torAddress, setTorAddress] = useState('');
const handleDismiss = () => { const handleDismiss = () => {
localStorage.setItem(STORAGE_KEY, 'true'); localStorage.setItem(STORAGE_KEY, 'true');
localStorage.setItem(LEGACY_STORAGE_KEY, 'true');
onClose(); onClose();
}; };
const handleOpenSettings = () => { const handleOpenSettings = () => {
localStorage.setItem(STORAGE_KEY, 'true'); localStorage.setItem(STORAGE_KEY, 'true');
localStorage.setItem(LEGACY_STORAGE_KEY, 'true');
onClose(); onClose();
onOpenSettings(); onOpenSettings();
}; };
const saveSetupKeys = async () => {
const payload = Object.fromEntries(
Object.entries(setupKeys).filter(([, value]) => value.trim()),
);
if (!Object.keys(payload).length) {
setSetupMsg({ type: 'err', text: 'Enter at least one API key first.' });
return;
}
setSetupSaving(true);
setSetupMsg(null);
try {
const res = await fetch('/api/settings/api-keys', {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload),
});
const data = await res.json().catch(() => ({}));
if (!res.ok || data?.ok === false) {
throw new Error(data?.detail || 'Could not save API keys.');
}
setSetupKeys({ OPENSKY_CLIENT_ID: '', OPENSKY_CLIENT_SECRET: '', AIS_API_KEY: '' });
setSetupMsg({ type: 'ok', text: 'Keys saved locally. Restart or refresh feeds to use them.' });
} catch (error) {
setSetupMsg({
type: 'err',
text: error instanceof Error ? error.message : 'Could not save API keys.',
});
} finally {
setSetupSaving(false);
}
};
const agentEndpoint =
agentMode === 'local'
? 'http://localhost:8000'
: torAddress || '<prepare remote .onion link>';
const agentSnippet = [
`SHADOWBROKER_URL=${agentEndpoint}`,
agentSecret ? `SHADOWBROKER_KEY=${agentSecret}` : 'SHADOWBROKER_KEY=<generate in ShadowBroker>',
`SHADOWBROKER_ACCESS=${agentTier}`,
'',
'# FIRST: load available tools',
`GET ${agentEndpoint}/api/ai/tools`,
'',
'# Auth: HMAC-SHA256 signed requests.',
'# Restricted = read-only telemetry. Full = can write when asked.',
].join('\n');
const remoteAgentNeedsTor = agentMode === 'remote' && !torAddress;
const fetchAgentConnectInfo = async (reveal = true) => {
setAgentLoading(true);
setAgentMsg(null);
try {
const res = await fetch(`/api/ai/connect-info?reveal=${reveal ? 'true' : 'false'}`);
const data = await res.json().catch(() => ({}));
if (!res.ok || data?.ok === false) {
throw new Error(data?.detail || 'Could not prepare agent credentials.');
}
setAgentSecret(data.hmac_secret || '');
setAgentTier(data.access_tier === 'full' ? 'full' : 'restricted');
setAgentMsg({ type: 'ok', text: 'Agent key is ready. Copy it into your local or remote agent runtime.' });
} catch (error) {
setAgentMsg({
type: 'err',
text: error instanceof Error ? error.message : 'Could not prepare agent credentials.',
});
} finally {
setAgentLoading(false);
}
};
const saveAgentTier = async (tier: 'restricted' | 'full') => {
setAgentTier(tier);
setAgentMsg(null);
try {
const res = await fetch('/api/ai/connect-info/access-tier', {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ tier }),
});
const data = await res.json().catch(() => ({}));
if (!res.ok || data?.ok === false) {
throw new Error(data?.detail || 'Could not update agent access tier.');
}
setAgentMsg({
type: 'ok',
text: tier === 'full'
? 'Full access saved. The agent can write to the dashboard when authenticated.'
: 'Restricted access saved. The agent can read telemetry but cannot write.',
});
} catch (error) {
setAgentMsg({
type: 'err',
text: error instanceof Error ? error.message : 'Could not update agent access tier.',
});
}
};
const prepareTorAgentAddress = async () => {
setTorStarting(true);
setAgentMsg(null);
try {
const res = await fetch('/api/settings/tor/start', { method: 'POST' });
const data = await res.json().catch(() => ({}));
if (!res.ok || data?.ok === false || !data?.onion_address) {
throw new Error(data?.detail || 'Could not start Tor hidden service.');
}
setTorAddress(data.onion_address);
setAgentMsg({
type: 'ok',
text: 'Tor is ready. The remote agent link is private to your local ShadowBroker node.',
});
} catch (error) {
setAgentMsg({
type: 'err',
text:
error instanceof Error
? error.message
: 'ShadowBroker could not install or start Tor automatically. Check network access and try again.',
});
} finally {
setTorStarting(false);
}
};
const copyAgentSnippet = async () => {
if (remoteAgentNeedsTor) {
setAgentMsg({ type: 'err', text: 'Install Tor and create the remote link first, then copy the agent config.' });
return;
}
await navigator.clipboard.writeText(agentSnippet);
setAgentCopied(true);
setTimeout(() => setAgentCopied(false), 1600);
};
return ( return (
<AnimatePresence> <AnimatePresence>
{/* Backdrop */} {/* Backdrop */}
@@ -123,7 +278,7 @@ const OnboardingModal = React.memo(function OnboardingModal({
{/* Step Indicators */} {/* Step Indicators */}
<div className="flex gap-2 px-6 pt-4"> <div className="flex gap-2 px-6 pt-4">
{['Welcome', 'API Keys', 'Free Sources'].map((label, i) => ( {['API Keys', 'AI Agent', 'Trust Modes', 'Free Sources'].map((label, i) => (
<button <button
key={label} key={label}
onClick={() => setStep(i)} onClick={() => setStep(i)}
@@ -140,36 +295,23 @@ const OnboardingModal = React.memo(function OnboardingModal({
{/* Content */} {/* Content */}
<div className="flex-1 overflow-y-auto styled-scrollbar p-6"> <div className="flex-1 overflow-y-auto styled-scrollbar p-6">
{step === 0 && ( {step === 2 && (
<div className="space-y-4"> <div className="space-y-4">
<div className="text-center py-4"> <div className="text-center py-4">
<div className="text-lg font-bold tracking-[0.3em] text-[var(--text-primary)] font-mono mb-2"> <div className="text-lg font-bold tracking-[0.3em] text-[var(--text-primary)] font-mono mb-2">
S H A D O W <span className="text-cyan-400">B R O K E R</span> T R U S T <span className="text-cyan-400">M O D E S</span>
</div> </div>
<p className="text-[11px] text-[var(--text-secondary)] font-mono leading-relaxed max-w-md mx-auto"> <p className="hidden">
Real-time OSINT dashboard aggregating 12+ live intelligence sources. Flights, Real-time OSINT dashboard aggregating 12+ live intelligence sources. Flights,
ships, satellites, earthquakes, conflicts, and more all on one map. ships, satellites, earthquakes, conflicts, and more all on one map.
</p> </p>
<p className="text-[11px] text-[var(--text-secondary)] font-mono leading-relaxed max-w-md mx-auto">
These modes explain what lane the network is using. Set up the API keys first,
then use this screen to understand public mesh versus private Wormhole paths.
</p>
</div> </div>
<div className="bg-yellow-950/20 border border-yellow-500/20 p-4"> <div className="hidden">
<div className="flex items-start gap-2">
<Key size={14} className="text-yellow-500 mt-0.5 flex-shrink-0" />
<div>
<p className="text-[11px] text-yellow-400 font-mono font-bold mb-1">
API Keys Required
</p>
<p className="text-sm text-[var(--text-secondary)] font-mono leading-relaxed">
Two API keys are needed for full functionality:{' '}
<span className="text-cyan-400">OpenSky Network</span> (flights) and{' '}
<span className="text-blue-400">AIS Stream</span> (ships). Both are free.
Without them, some panels will show no data.
</p>
</div>
</div>
</div>
<div className="bg-green-950/20 border border-green-500/20 p-4">
<div className="flex items-start gap-2"> <div className="flex items-start gap-2">
<Globe size={14} className="text-green-500 mt-0.5 flex-shrink-0" /> <Globe size={14} className="text-green-500 mt-0.5 flex-shrink-0" />
<div> <div>
@@ -217,7 +359,219 @@ const OnboardingModal = React.memo(function OnboardingModal({
)} )}
{step === 1 && ( {step === 1 && (
<div className="space-y-5">
<div>
<p className="text-[11px] text-violet-300 font-mono font-bold tracking-widest mb-2">
STEP 1 - WHERE IS YOUR AGENT?
</p>
<div className="grid grid-cols-2 gap-2">
<button
onClick={() => setAgentMode('local')}
className={`border px-4 py-3 text-left transition-all ${
agentMode === 'local'
? 'border-cyan-500/50 bg-cyan-950/40'
: 'border-[var(--border-primary)] hover:border-cyan-500/30'
}`}
>
<p className={`text-sm font-mono font-bold ${agentMode === 'local' ? 'text-cyan-300' : 'text-[var(--text-secondary)]'}`}>
Local
</p>
<p className="text-[10px] text-[var(--text-muted)] font-mono mt-1">
Same machine as ShadowBroker
</p>
</button>
<button
onClick={() => setAgentMode('remote')}
className={`border px-4 py-3 text-left transition-all ${
agentMode === 'remote'
? 'border-violet-500/50 bg-violet-950/40'
: 'border-[var(--border-primary)] hover:border-violet-500/30'
}`}
>
<p className={`text-sm font-mono font-bold ${agentMode === 'remote' ? 'text-violet-300' : 'text-[var(--text-secondary)]'}`}>
Remote
</p>
<p className="text-[10px] text-[var(--text-muted)] font-mono mt-1">
Different machine over Tor
</p>
</button>
</div>
</div>
<div>
<p className="text-[11px] text-violet-300 font-mono font-bold tracking-widest mb-2">
STEP 2 - WHAT CAN IT DO?
</p>
<div className="grid grid-cols-2 gap-2">
<button
onClick={() => void saveAgentTier('restricted')}
className={`border px-4 py-3 text-left transition-all ${
agentTier === 'restricted'
? 'border-green-500/50 bg-green-950/30'
: 'border-[var(--border-primary)] hover:border-green-500/30'
}`}
>
<p className="text-sm text-green-300 font-mono font-bold flex items-center gap-2">
<Shield size={14} /> Read Only
</p>
<p className="text-[10px] text-[var(--text-muted)] font-mono mt-2">
Can see live telemetry but cannot change anything
</p>
</button>
<button
onClick={() => void saveAgentTier('full')}
className={`border px-4 py-3 text-left transition-all ${
agentTier === 'full'
? 'border-amber-500/50 bg-amber-950/30'
: 'border-[var(--border-primary)] hover:border-amber-500/30'
}`}
>
<p className="text-sm text-amber-300 font-mono font-bold flex items-center gap-2">
<Network size={14} /> Full Access
</p>
<p className="text-[10px] text-[var(--text-muted)] font-mono mt-2">
Can place pins, create layers, and trigger display actions
</p>
</button>
</div>
</div>
<div>
<div className="flex items-center justify-between gap-3 mb-2">
<div>
<p className="text-[11px] text-violet-300 font-mono font-bold tracking-widest">
STEP 3 - COPY THIS INTO YOUR AGENT
</p>
<p className="text-[10px] text-[var(--text-muted)] font-mono mt-1">
Generate a local key, then copy these variables into OpenClaw, Hermes, or another HMAC agent.
</p>
</div>
<button
onClick={() => void fetchAgentConnectInfo(true)}
disabled={agentLoading}
className="px-3 py-2 border border-violet-500/40 text-violet-300 hover:bg-violet-500/10 disabled:opacity-50 text-[11px] font-mono tracking-widest"
>
{agentLoading ? 'GENERATING...' : 'GENERATE'}
</button>
</div>
{remoteAgentNeedsTor && (
<div className="mb-2 border border-violet-500/30 bg-violet-950/20 p-3">
<div className="flex items-start justify-between gap-3">
<div>
<p className="text-[11px] text-violet-200 font-mono font-bold tracking-widest">
TOR REQUIRED FOR REMOTE AGENTS
</p>
<p className="text-[10px] text-[var(--text-muted)] font-mono mt-1 leading-relaxed">
ShadowBroker will install or use Tor locally, then create a private .onion link for this backend.
</p>
</div>
<button
onClick={() => void prepareTorAgentAddress()}
disabled={torStarting}
className="shrink-0 px-3 py-2 border border-violet-500/40 text-violet-200 hover:bg-violet-500/10 disabled:opacity-50 text-[10px] font-mono tracking-widest flex items-center gap-2"
>
<Network size={13} />
{torStarting ? 'INSTALLING...' : 'INSTALL TOR'}
</button>
</div>
</div>
)}
<div className="relative">
<pre className="min-h-40 max-h-56 overflow-auto styled-scrollbar bg-[var(--bg-primary)] border border-violet-500/30 p-4 pr-24 text-[12px] text-violet-100 font-mono whitespace-pre-wrap leading-relaxed">
{agentSnippet}
</pre>
<button
onClick={() => void copyAgentSnippet()}
disabled={remoteAgentNeedsTor}
className="absolute top-3 right-3 px-3 py-2 border border-violet-500/50 bg-violet-950/50 text-violet-200 hover:bg-violet-800/30 disabled:opacity-45 disabled:hover:bg-violet-950/50 text-[11px] font-mono tracking-widest flex items-center gap-2"
>
{agentCopied ? <Check size={13} /> : <Copy size={13} />}
{agentCopied ? 'COPIED' : 'COPY'}
</button>
</div>
{agentMsg && (
<p
className={`mt-2 text-sm font-mono ${
agentMsg.type === 'ok' ? 'text-green-300' : 'text-red-300'
}`}
>
{agentMsg.text}
</p>
)}
</div>
<p className="text-[11px] text-orange-300/80 font-mono leading-relaxed">
Remote agent access uses the signed HTTP API over Tor. Wormhole uses the same Tor/Arti transport lane when it is available; MLS-native agent transport is still planned.
</p>
</div>
)}
{step === 0 && (
<div className="space-y-4"> <div className="space-y-4">
<div className="bg-yellow-950/20 border border-yellow-500/20 p-4">
<div className="flex items-start gap-2">
<Key size={14} className="text-yellow-500 mt-0.5 flex-shrink-0" />
<div>
<p className="text-[11px] text-yellow-400 font-mono font-bold mb-1">
START HERE
</p>
<p className="text-sm text-[var(--text-secondary)] font-mono leading-relaxed">
OpenSky Network and AIS Stream are the free keys that make ShadowBroker
useful immediately: live aircraft and vessel tracking. Paste them below or
use Settings later; secrets stay on the local backend.
</p>
</div>
</div>
</div>
<div className="border border-cyan-900/40 bg-cyan-950/10 p-4 space-y-3">
<div>
<p className="text-[11px] text-cyan-300 font-mono font-bold tracking-widest">
QUICK LOCAL SETUP
</p>
<p className="text-sm text-[var(--text-secondary)] font-mono leading-relaxed mt-1">
Paste keys here once. ShadowBroker stores them server-side only and never
displays the secret back in the browser.
</p>
</div>
{[
['OPENSKY_CLIENT_ID', 'OpenSky Client ID'],
['OPENSKY_CLIENT_SECRET', 'OpenSky Client Secret'],
['AIS_API_KEY', 'AIS Stream API Key'],
].map(([key, label]) => (
<input
key={key}
type="password"
value={setupKeys[key as keyof typeof setupKeys]}
onChange={(event) =>
setSetupKeys((prev) => ({ ...prev, [key]: event.target.value }))
}
placeholder={label}
className="w-full bg-[var(--bg-primary)] border border-[var(--border-primary)] px-3 py-2 text-sm text-[var(--text-primary)] font-mono outline-none focus:border-cyan-500/70 placeholder:text-[var(--text-muted)]/60"
autoComplete="off"
/>
))}
{setupMsg && (
<p
className={`text-sm font-mono ${
setupMsg.type === 'ok' ? 'text-green-300' : 'text-red-300'
}`}
>
{setupMsg.text}
</p>
)}
<button
onClick={() => void saveSetupKeys()}
disabled={setupSaving}
className="w-full py-2 bg-cyan-500/10 border border-cyan-500/30 text-cyan-400 hover:bg-cyan-500/20 disabled:opacity-50 disabled:cursor-not-allowed transition-colors text-[11px] font-mono tracking-widest"
>
{setupSaving ? 'SAVING...' : 'SAVE KEYS LOCALLY'}
</button>
</div>
{API_GUIDES.map((api) => ( {API_GUIDES.map((api) => (
<div <div
key={api.name} key={api.name}
@@ -268,11 +622,12 @@ const OnboardingModal = React.memo(function OnboardingModal({
</div> </div>
)} )}
{step === 2 && ( {step === 3 && (
<div className="space-y-3"> <div className="space-y-3">
<p className="text-sm text-[var(--text-secondary)] font-mono mb-3"> <p className="text-sm text-[var(--text-secondary)] font-mono mb-3">
These data sources are completely free and require no API keys. They activate These data sources are completely free and require no API keys. They activate
automatically on launch. automatically on launch, while OpenSky and AIS Stream unlock the richer live
aviation and maritime experience.
</p> </p>
<div className="grid grid-cols-2 gap-2"> <div className="grid grid-cols-2 gap-2">
{FREE_SOURCES.map((src) => ( {FREE_SOURCES.map((src) => (
@@ -309,7 +664,7 @@ const OnboardingModal = React.memo(function OnboardingModal({
</button> </button>
<div className="flex gap-1.5"> <div className="flex gap-1.5">
{[0, 1, 2].map((i) => ( {[0, 1, 2, 3].map((i) => (
<div <div
key={i} key={i}
className={`w-1.5 h-1.5 rounded-full transition-colors ${step === i ? 'bg-cyan-400' : 'bg-[var(--border-primary)]'}`} className={`w-1.5 h-1.5 rounded-full transition-colors ${step === i ? 'bg-cyan-400' : 'bg-[var(--border-primary)]'}`}
@@ -317,7 +672,7 @@ const OnboardingModal = React.memo(function OnboardingModal({
))} ))}
</div> </div>
{step < 2 ? ( {step < 3 ? (
<button <button
onClick={() => setStep(step + 1)} onClick={() => setStep(step + 1)}
className="px-4 py-2 border border-cyan-500/40 text-cyan-400 hover:bg-cyan-500/10 text-sm font-mono tracking-widest transition-all" className="px-4 py-2 border border-cyan-500/40 text-cyan-400 hover:bg-cyan-500/10 text-sm font-mono tracking-widest transition-all"
+163 -35
View File
@@ -120,6 +120,9 @@ interface EnvMeta {
env_path_writable: boolean; env_path_writable: boolean;
env_example_path: string; env_example_path: string;
env_example_path_exists: boolean; env_example_path_exists: boolean;
operator_keys_env_path?: string;
operator_keys_env_path_exists?: boolean;
operator_keys_env_path_writable?: boolean;
} }
const WEIGHT_LABELS: Record<number, string> = { const WEIGHT_LABELS: Record<number, string> = {
@@ -493,10 +496,13 @@ const SettingsPanel = React.memo(function SettingsPanel({
}, [adminKey, refreshAdminSession]); }, [adminKey, refreshAdminSession]);
// --- API Keys state --- // --- API Keys state ---
// API keys are intentionally NOT editable in-app. The panel is read-only and // API keys are write-only in-app. Values are sent once to the local backend,
// tells the user where the .env file lives so they can edit it directly. // stored server-side, and never returned to the browser.
// This keeps secrets off the wire and out of the browser process.
const [apis, setApis] = useState<ApiEntry[]>([]); const [apis, setApis] = useState<ApiEntry[]>([]);
const [apiKeyInputs, setApiKeyInputs] = useState<Record<string, string>>({});
const [apiKeyEditing, setApiKeyEditing] = useState<Record<string, boolean>>({});
const [apiKeySaving, setApiKeySaving] = useState<string | null>(null);
const [apiKeyMsg, setApiKeyMsg] = useState<{ type: 'ok' | 'err'; text: string } | null>(null);
const [expandedCategories, setExpandedCategories] = useState<Set<string>>( const [expandedCategories, setExpandedCategories] = useState<Set<string>>(
new Set(['Aviation', 'Maritime']), new Set(['Aviation', 'Maritime']),
); );
@@ -535,7 +541,9 @@ const SettingsPanel = React.memo(function SettingsPanel({
const fetchKeys = useCallback(async () => { const fetchKeys = useCallback(async () => {
try { try {
setApis(await controlPlaneJson<ApiEntry[]>('/api/settings/api-keys')); setApis(await controlPlaneJson<ApiEntry[]>('/api/settings/api-keys', {
requireAdminSession: false,
}));
return true; return true;
} catch (e) { } catch (e) {
await handleProtectedSettingsError(e); await handleProtectedSettingsError(e);
@@ -543,6 +551,41 @@ const SettingsPanel = React.memo(function SettingsPanel({
} }
}, [handleProtectedSettingsError]); }, [handleProtectedSettingsError]);
const saveApiKey = useCallback(
async (envKey: string | null) => {
if (!envKey) return;
const value = String(apiKeyInputs[envKey] || '').trim();
if (!value) {
setApiKeyMsg({ type: 'err', text: `Enter a value for ${envKey}.` });
return;
}
setApiKeySaving(envKey);
setApiKeyMsg(null);
try {
const result = await controlPlaneJson<{
keys?: ApiEntry[];
env?: EnvMeta;
}>('/api/settings/api-keys', {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ [envKey]: value }),
requireAdminSession: false,
});
if (result.keys) setApis(result.keys);
if (result.env) setEnvMeta(result.env);
setApiKeyInputs((prev) => ({ ...prev, [envKey]: '' }));
setApiKeyEditing((prev) => ({ ...prev, [envKey]: false }));
setApiKeyMsg({ type: 'ok', text: `${envKey} saved locally. Restart or refresh feeds to use it.` });
} catch (e) {
const message = e instanceof Error ? e.message : 'Could not save API key';
setApiKeyMsg({ type: 'err', text: message });
} finally {
setApiKeySaving(null);
}
},
[apiKeyInputs],
);
const fetchEnvMeta = useCallback(async () => { const fetchEnvMeta = useCallback(async () => {
try { try {
const res = await fetch('/api/settings/api-keys/meta'); const res = await fetch('/api/settings/api-keys/meta');
@@ -663,10 +706,10 @@ const SettingsPanel = React.memo(function SettingsPanel({
} }
void (async () => { void (async () => {
const ready = await refreshAdminSession(); const ready = await refreshAdminSession();
await fetchKeys();
if (ready) { if (ready) {
await Promise.all([fetchKeys(), fetchFeeds()]); await fetchFeeds();
} else { } else {
setApis([]);
setFeeds([]); setFeeds([]);
setFeedsDirty(false); setFeedsDirty(false);
} }
@@ -713,12 +756,13 @@ const SettingsPanel = React.memo(function SettingsPanel({
}, [onClose, wormholeEnabled, wormholeSaving, wormholeStatus]); }, [onClose, wormholeEnabled, wormholeSaving, wormholeStatus]);
useEffect(() => { useEffect(() => {
if (!isOpen || !adminSessionReady) return; if (!isOpen) return;
if (activeTab === 'api-keys') { if (activeTab === 'api-keys') {
void fetchKeys(); void fetchKeys();
void fetchEnvMeta(); void fetchEnvMeta();
return; return;
} }
if (!adminSessionReady) return;
if (activeTab === 'news-feeds') { if (activeTab === 'news-feeds') {
void fetchFeeds(); void fetchFeeds();
} }
@@ -2166,18 +2210,25 @@ const SettingsPanel = React.memo(function SettingsPanel({
<div className="flex items-start gap-2"> <div className="flex items-start gap-2">
<Shield size={12} className="text-cyan-500 mt-0.5 flex-shrink-0" /> <Shield size={12} className="text-cyan-500 mt-0.5 flex-shrink-0" />
<p className="text-sm text-[var(--text-secondary)] font-mono leading-relaxed"> <p className="text-sm text-[var(--text-secondary)] font-mono leading-relaxed">
API keys are stored locally in the backend{' '} API keys are saved locally by this backend. Values are write-only: the app
<span className="text-cyan-400">.env</span> file. Keys marked with{' '} stores the key and shows CONFIGURED, but it never reads the secret back into
<Key size={8} className="inline text-yellow-500" /> are required for full the browser. Keys marked with{' '}
functionality. Public APIs need no key. <Key size={8} className="inline text-yellow-500" /> unlock the richest live
aircraft and vessel feeds.
</p> </p>
</div> </div>
<div className="pl-5 text-[12px] font-mono text-cyan-200/80 leading-relaxed">
Configured keys stay hidden for shared dashboards. Unlock operator tools, then
use ROTATE only when you intentionally want to replace a working credential.
</div>
{envMeta && ( {envMeta && (
<div className="pl-5 text-[12px] font-mono text-[var(--text-muted)] leading-relaxed space-y-0.5"> <div className="pl-5 text-[12px] font-mono text-[var(--text-muted)] leading-relaxed space-y-0.5">
<div> <div>
<span className="text-cyan-500/70">.env path:</span>{' '} <span className="text-cyan-500/70">local key store:</span>{' '}
<span className="text-cyan-300 break-all select-all">{envMeta.env_path}</span>{' '} <span className="text-cyan-300 break-all select-all">
{envMeta.env_path_exists ? ( {envMeta.operator_keys_env_path || envMeta.env_path}
</span>{' '}
{envMeta.operator_keys_env_path_exists || envMeta.env_path_exists ? (
<span className="text-green-400/80">[exists]</span> <span className="text-green-400/80">[exists]</span>
) : ( ) : (
<span className="text-amber-400/80">[will be created on first save]</span> <span className="text-amber-400/80">[will be created on first save]</span>
@@ -2199,6 +2250,15 @@ const SettingsPanel = React.memo(function SettingsPanel({
)} )}
</div> </div>
)} )}
{apiKeyMsg && (
<div
className={`pl-5 text-sm font-mono ${
apiKeyMsg.type === 'ok' ? 'text-green-300' : 'text-red-300'
}`}
>
{apiKeyMsg.text}
</div>
)}
</div> </div>
{/* API List */} {/* API List */}
@@ -2288,33 +2348,101 @@ const SettingsPanel = React.memo(function SettingsPanel({
{api.description} {api.description}
</p> </p>
{api.has_key && ( {api.has_key && (
<div className="mt-2 flex items-center gap-2 text-[12px] font-mono"> <div className="mt-2 space-y-2 text-[12px] font-mono">
{api.is_set ? ( {api.is_set ? (
<> <div className="space-y-2">
<span className="px-2 py-0.5 border border-green-500/40 bg-green-950/20 text-green-300 tracking-wider"> <div className="flex items-start justify-between gap-2">
CONFIGURED <div className="min-w-0 flex items-center gap-2">
</span> <span className="px-2 py-0.5 border border-green-500/40 bg-green-950/20 text-green-300 tracking-wider">
<span className="text-[var(--text-muted)]"> CONFIGURED
edit{' '} </span>
<span className="text-cyan-300 select-all break-all"> <span className="text-[var(--text-muted)] leading-relaxed">
{api.env_key} Secret hidden. Stored write-only on this backend as{' '}
</span>{' '} <span className="text-cyan-300 select-all break-all">
in the .env file (path shown above) and restart the backend. {api.env_key}
</span> </span>
</> .
</span>
</div>
{api.env_key && (
<button
type="button"
onClick={() => {
if (!(nativeProtected || adminSessionReady)) {
setApiKeyMsg({
type: 'err',
text: 'Unlock operator tools before rotating a configured key.',
});
return;
}
setApiKeyMsg(null);
setApiKeyEditing((prev) => ({
...prev,
[api.env_key as string]: !prev[api.env_key as string],
}));
}}
className={`shrink-0 px-2 py-1 border text-[11px] tracking-widest transition-colors ${
nativeProtected || adminSessionReady
? 'border-yellow-500/40 text-yellow-300 hover:bg-yellow-500/10'
: 'border-[var(--border-primary)] text-[var(--text-muted)] hover:border-yellow-500/30 hover:text-yellow-300/80'
}`}
>
{apiKeyEditing[api.env_key] ? 'CANCEL' : 'ROTATE'}
</button>
)}
</div>
{!(nativeProtected || adminSessionReady) && (
<div className="text-[11px] text-yellow-300/70 leading-relaxed">
Operator tools are locked. Viewers can see source status
but cannot replace saved credentials.
</div>
)}
</div>
) : ( ) : (
<> <div className="flex items-center gap-2">
<span className="px-2 py-0.5 border border-amber-500/40 bg-amber-950/20 text-amber-300 tracking-wider"> <span className="px-2 py-0.5 border border-amber-500/40 bg-amber-950/20 text-amber-300 tracking-wider">
NOT CONFIGURED NOT CONFIGURED
</span> </span>
<span className="text-[var(--text-muted)]"> <span className="text-[var(--text-muted)]">
add{' '} Save {api.env_key} here to enable this source.
<span className="text-amber-200 select-all break-all">
{api.env_key}=YOUR_VALUE
</span>{' '}
to the .env file (path shown above) and restart the backend.
</span> </span>
</> </div>
)}
{(!api.is_set || (api.env_key && apiKeyEditing[api.env_key])) && (
<div className="flex items-center gap-2">
<input
type="password"
value={api.env_key ? apiKeyInputs[api.env_key] || '' : ''}
onChange={(event) => {
if (!api.env_key) return;
setApiKeyInputs((prev) => ({
...prev,
[api.env_key as string]: event.target.value,
}));
}}
placeholder={
api.is_set
? 'Enter replacement key...'
: `Enter ${api.env_key}...`
}
className="min-w-0 flex-1 bg-[var(--bg-primary)] border border-[var(--border-primary)] px-2 py-1.5 text-sm text-[var(--text-primary)] outline-none focus:border-cyan-500/70 placeholder:text-[var(--text-muted)]/50"
autoComplete="off"
/>
<button
onClick={() => void saveApiKey(api.env_key)}
disabled={
!api.env_key ||
apiKeySaving === api.env_key ||
!String(
api.env_key ? apiKeyInputs[api.env_key] || '' : '',
).trim()
}
className="h-8 px-3 border border-cyan-500/40 bg-cyan-950/20 text-cyan-300 hover:bg-cyan-500/15 disabled:opacity-40 disabled:cursor-not-allowed flex items-center gap-1.5 tracking-widest"
>
<Save size={12} />
{apiKeySaving === api.env_key ? 'SAVING' : 'SAVE'}
</button>
</div>
)} )}
</div> </div>
)} )}
@@ -2332,7 +2460,7 @@ const SettingsPanel = React.memo(function SettingsPanel({
<div className="p-4 border-t border-[var(--border-primary)]/80"> <div className="p-4 border-t border-[var(--border-primary)]/80">
<div className="flex items-center justify-between text-[13px] text-[var(--text-muted)] font-mono"> <div className="flex items-center justify-between text-[13px] text-[var(--text-muted)] font-mono">
<span>{apis.length} REGISTERED APIs</span> <span>{apis.length} REGISTERED APIs</span>
<span>{apis.filter((a) => a.has_key).length} KEYS CONFIGURED</span> <span>{apis.filter((a) => a.has_key && a.is_set).length} KEYS CONFIGURED</span>
</div> </div>
</div> </div>
</> </>
@@ -0,0 +1,106 @@
'use client';
import React, { useEffect, useState } from 'react';
import { motion, AnimatePresence } from 'framer-motion';
import { Database, Clock, X } from 'lucide-react';
const CURRENT_VERSION = '0.9.75';
const STORAGE_KEY = `shadowbroker_startup_warmup_notice_v${CURRENT_VERSION}`;
interface StartupWarmupModalProps {
onClose: () => void;
}
export default function StartupWarmupModal({ onClose }: StartupWarmupModalProps) {
const handleDismiss = () => {
localStorage.setItem(STORAGE_KEY, 'true');
onClose();
};
return (
<AnimatePresence>
<motion.div
key="warmup-backdrop"
initial={{ opacity: 0 }}
animate={{ opacity: 1 }}
exit={{ opacity: 0 }}
className="fixed inset-0 bg-black/80 backdrop-blur-sm z-[10000]"
onClick={handleDismiss}
/>
<motion.div
key="warmup-modal"
initial={{ opacity: 0, scale: 0.92, y: 18 }}
animate={{ opacity: 1, scale: 1, y: 0 }}
exit={{ opacity: 0, scale: 0.92, y: 18 }}
transition={{ type: 'spring', damping: 25, stiffness: 300 }}
className="fixed inset-0 z-[10001] flex items-center justify-center pointer-events-none"
>
<div
className="w-[520px] max-w-[calc(100vw-32px)] bg-[var(--bg-secondary)]/98 border border-cyan-900/50 pointer-events-auto overflow-hidden"
onClick={(e) => e.stopPropagation()}
>
<div className="p-5 border-b border-[var(--border-primary)]/80 flex items-center justify-between">
<div className="flex items-center gap-3">
<div className="w-10 h-10 bg-cyan-500/10 border border-cyan-500/30 flex items-center justify-center">
<Database size={18} className="text-cyan-400" />
</div>
<div>
<h2 className="text-sm font-bold tracking-[0.2em] text-[var(--text-primary)] font-mono">
STARTUP CACHE
</h2>
<span className="text-[13px] text-[var(--text-muted)] font-mono tracking-widest">
FIRST RUN WARMUP
</span>
</div>
</div>
<button
onClick={handleDismiss}
className="w-8 h-8 border border-[var(--border-primary)] hover:border-red-500/50 flex items-center justify-center text-[var(--text-muted)] hover:text-red-400 transition-all hover:bg-red-950/20"
>
<X size={14} />
</button>
</div>
<div className="p-5 space-y-4">
<div className="bg-cyan-950/20 border border-cyan-500/20 p-4">
<div className="flex items-start gap-3">
<Clock size={15} className="text-cyan-400 mt-0.5 flex-shrink-0" />
<div className="space-y-2">
<p className="text-[11px] text-cyan-300 font-mono font-bold tracking-widest">
MASS DATA SYNTHESIS
</p>
<p className="text-sm text-[var(--text-secondary)] font-mono leading-relaxed">
The first launch builds local caches for flights, ships, satellites, CCTV, fires,
and threat intelligence. Cached launches paint the map much faster; a brand-new
install can take a few minutes while upstream feeds are synthesized.
</p>
</div>
</div>
</div>
<button
onClick={handleDismiss}
className="w-full py-3 border border-cyan-500/40 text-cyan-300 hover:text-cyan-100 hover:border-cyan-400/70 hover:bg-cyan-950/30 transition-all font-mono text-[12px] tracking-[0.18em] font-bold"
>
CONTINUE
</button>
</div>
</div>
</motion.div>
</AnimatePresence>
);
}
export function useStartupWarmupNotice() {
const [showWarmupNotice, setShowWarmupNotice] = useState(false);
useEffect(() => {
try {
setShowWarmupNotice(localStorage.getItem(STORAGE_KEY) !== 'true');
} catch {
setShowWarmupNotice(false);
}
}, []);
return { showWarmupNotice, setShowWarmupNotice };
}
+13 -9
View File
@@ -33,8 +33,8 @@ import {
import { purgeBrowserContactGraph, purgeBrowserSigningMaterial, setSecureModeCached, getNodeIdentity, generateNodeKeys } from '@/mesh/meshIdentity'; import { purgeBrowserContactGraph, purgeBrowserSigningMaterial, setSecureModeCached, getNodeIdentity, generateNodeKeys } from '@/mesh/meshIdentity';
import { purgeBrowserDmState } from '@/mesh/meshDmWorkerClient'; import { purgeBrowserDmState } from '@/mesh/meshDmWorkerClient';
import { import {
DEFAULT_INFONET_SEED_URL,
fetchInfonetNodeStatusSnapshot, fetchInfonetNodeStatusSnapshot,
startTorHiddenService,
type InfonetNodeStatusSnapshot, type InfonetNodeStatusSnapshot,
} from '@/mesh/controlPlaneStatusClient'; } from '@/mesh/controlPlaneStatusClient';
import { import {
@@ -263,6 +263,10 @@ export default function TopRightControls({
await generateNodeKeys(); await generateNodeKeys();
} }
setActivatingPhase('peers'); setActivatingPhase('peers');
const torStatus = await startTorHiddenService();
if (!torStatus?.running || !torStatus?.onion_address) {
throw new Error(torStatus?.detail || 'Tor onion service did not start');
}
} }
const res = await controlPlaneFetch('/api/settings/node', { const res = await controlPlaneFetch('/api/settings/node', {
@@ -833,9 +837,9 @@ export default function TopRightControls({
: activatingPhase === 'peers' ? 'text-cyan-300' : activatingPhase === 'peers' ? 'text-cyan-300'
: 'text-green-300' : 'text-green-300'
}> }>
{activatingPhase === 'keys' ? 'Connecting to default seed...' {activatingPhase === 'keys' ? 'Preparing onion transport...'
: activatingPhase === 'peers' ? 'Connecting to default seed...' : activatingPhase === 'peers' ? 'Finding bootstrap peers...'
: 'Default seed connected'} : 'Bootstrap peers ready'}
</span> </span>
</div> </div>
{/* Step: Sync chain */} {/* Step: Sync chain */}
@@ -899,7 +903,7 @@ export default function TopRightControls({
<div className="border border-cyan-500/20 bg-cyan-950/10 px-4 py-4 text-[10px] font-mono text-cyan-100 leading-[1.8]"> <div className="border border-cyan-500/20 bg-cyan-950/10 px-4 py-4 text-[10px] font-mono text-cyan-100 leading-[1.8]">
Do you want to activate a node on this install? Do you want to activate a node on this install?
<div className="mt-2 text-[9px] text-cyan-200/70 normal-case tracking-normal"> <div className="mt-2 text-[9px] text-cyan-200/70 normal-case tracking-normal">
This turns on your local participant node and lets this install keep syncing the public Infonet chain from <span className="text-cyan-300">{DEFAULT_INFONET_SEED_URL}</span>. This turns on your local participant node and syncs Infonet only through available Wormhole onion/RNS peers. Clearnet bootstrap is disabled by default.
</div> </div>
</div> </div>
{(bootstrapFailed || nodeStatusError || nodeToggleError) && ( {(bootstrapFailed || nodeStatusError || nodeToggleError) && (
@@ -930,10 +934,10 @@ export default function TopRightControls({
<div className="text-cyan-300 tracking-[0.18em]">BY CONTINUING YOU AGREE:</div> <div className="text-cyan-300 tracking-[0.18em]">BY CONTINUING YOU AGREE:</div>
<ul className="mt-3 space-y-2 list-disc pl-5"> <ul className="mt-3 space-y-2 list-disc pl-5">
<li>This install can keep a local copy of the public Infonet chain.</li> <li>This install can keep a local copy of the public Infonet chain.</li>
<li>Fresh installs pull from the bundled default seed at {DEFAULT_INFONET_SEED_URL}.</li> <li>Fresh installs do not use a clearnet Infonet seed.</li>
<li>Participant-node sync is public-facing unless you separately use obfuscated-lane features.</li> <li>Participant-node sync requires an onion/RNS peer through Wormhole.</li>
<li>Your backend may sync with configured or bundled bootstrap peers in the background.</li> <li>Your backend may sync with configured private bootstrap peers in the background.</li>
<li>Wormhole provides gates (transitional private lane) and Dead Drop / DM (stronger private lane) as separate postures.</li> <li>Wormhole keeps Infonet, gates, Dead Drop, and DM traffic on the obfuscated lane.</li>
</ul> </ul>
</div> </div>
<div className="text-[11px] font-mono uppercase tracking-[0.2em] text-cyan-300/80"> <div className="text-[11px] font-mono uppercase tracking-[0.2em] text-cyan-300/80">
@@ -34,7 +34,7 @@ export function useImperativeSource(
}; };
const pushWhenReady = () => { const pushWhenReady = () => {
let attemptsRemaining = 20; let attemptsRemaining = 150;
const tryPush = () => { const tryPush = () => {
if (cancelled) return; if (cancelled) return;
@@ -62,6 +62,7 @@ export function useImperativeSource(
pushWhenReady(); pushWhenReady();
}; };
rawMap.on('load', handleStyleData);
rawMap.on('styledata', handleStyleData); rawMap.on('styledata', handleStyleData);
// Skip redundant writes for unchanged references, but keep the styledata // Skip redundant writes for unchanged references, but keep the styledata
@@ -73,6 +74,7 @@ export function useImperativeSource(
return () => { return () => {
cancelled = true; cancelled = true;
rawMap.off('load', handleStyleData);
rawMap.off('styledata', handleStyleData); rawMap.off('styledata', handleStyleData);
if (timerRef.current) clearTimeout(timerRef.current); if (timerRef.current) clearTimeout(timerRef.current);
if (retryTimerRef.current) clearTimeout(retryTimerRef.current); if (retryTimerRef.current) clearTimeout(retryTimerRef.current);
+3 -1
View File
@@ -35,6 +35,7 @@ interface AgentAction {
export function useAgentActions( export function useAgentActions(
onShowImage: (coords: { lat: number; lng: number }) => void, onShowImage: (coords: { lat: number; lng: number }) => void,
onFlyTo?: (coords: { lat: number; lng: number; zoom?: number }) => void, onFlyTo?: (coords: { lat: number; lng: number; zoom?: number }) => void,
enabled = true,
) { ) {
const onShowImageRef = useRef(onShowImage); const onShowImageRef = useRef(onShowImage);
onShowImageRef.current = onShowImage; onShowImageRef.current = onShowImage;
@@ -70,9 +71,10 @@ export function useAgentActions(
useEffect(() => { useEffect(() => {
// Poll every 3 seconds — lightweight endpoint, ~50 bytes when empty // Poll every 3 seconds — lightweight endpoint, ~50 bytes when empty
if (!enabled) return;
const interval = setInterval(poll, 3000); const interval = setInterval(poll, 3000);
// Initial poll on mount // Initial poll on mount
poll(); poll();
return () => clearInterval(interval); return () => clearInterval(interval);
}, [poll]); }, [enabled, poll]);
} }
+46 -8
View File
@@ -59,6 +59,8 @@ type FastDataProbe = {
ships?: unknown[]; ships?: unknown[];
sigint?: unknown[]; sigint?: unknown[];
cctv?: unknown[]; cctv?: unknown[];
news?: unknown[];
threat_level?: unknown;
}; };
function hasMeaningfulFastData(json: FastDataProbe): boolean { function hasMeaningfulFastData(json: FastDataProbe): boolean {
@@ -100,11 +102,37 @@ export function useDataPolling() {
_slowEtagRef = slowEtag; _slowEtagRef = slowEtag;
let hasData = false; let hasData = false;
let fetchedStartupFastPayload = false;
let fastTimerId: ReturnType<typeof setTimeout> | null = null; let fastTimerId: ReturnType<typeof setTimeout> | null = null;
let slowTimerId: ReturnType<typeof setTimeout> | null = null; let slowTimerId: ReturnType<typeof setTimeout> | null = null;
const fastAbortRef = { current: null as AbortController | null }; const fastAbortRef = { current: null as AbortController | null };
const slowAbortRef = { current: null as AbortController | null }; const slowAbortRef = { current: null as AbortController | null };
const fetchCriticalBootstrap = async () => {
try {
const res = await fetch(`${API_BASE}/api/bootstrap/critical`, {
headers: { Accept: 'application/json' },
});
if (res.ok) {
setStoreBackendStatus('connected');
const json = await res.json();
mergeData(json);
if (hasMeaningfulFastData(json) || (json.news?.length || 0) > 0 || json.threat_level) {
hasData = true;
}
}
} catch (e) {
const aborted =
typeof e === 'object' &&
e !== null &&
'name' in e &&
(e as { name?: string }).name === 'AbortError';
if (!aborted) {
console.warn("Critical bootstrap fetch will retry via live polling", e);
}
}
};
const fetchFastData = async () => { const fetchFastData = async () => {
if (fastTimerId) { if (fastTimerId) {
clearTimeout(fastTimerId); clearTimeout(fastTimerId);
@@ -116,9 +144,11 @@ export function useDataPolling() {
const controller = new AbortController(); const controller = new AbortController();
fastAbortRef.current = controller; fastAbortRef.current = controller;
try { try {
const useStartupPayload = !fetchedStartupFastPayload && !fastEtag.current;
const headers: Record<string, string> = {}; const headers: Record<string, string> = {};
if (fastEtag.current) headers['If-None-Match'] = fastEtag.current; if (!useStartupPayload && fastEtag.current) headers['If-None-Match'] = fastEtag.current;
const res = await fetch(`${API_BASE}/api/live-data/fast`, { const url = `${API_BASE}/api/live-data/fast${useStartupPayload ? '?initial=1' : ''}`;
const res = await fetch(url, {
headers, headers,
signal: controller.signal, signal: controller.signal,
}); });
@@ -129,7 +159,10 @@ export function useDataPolling() {
} }
if (res.ok) { if (res.ok) {
setStoreBackendStatus('connected'); setStoreBackendStatus('connected');
fastEtag.current = res.headers.get('etag') || null; // Do not keep the capped startup ETag. The next steady poll should
// request the full fast dataset and replace the representative first paint.
fastEtag.current = useStartupPayload ? null : res.headers.get('etag') || null;
if (useStartupPayload) fetchedStartupFastPayload = true;
const json = await res.json(); const json = await res.json();
mergeData(json); mergeData(json);
if (hasMeaningfulFastData(json)) hasData = true; if (hasMeaningfulFastData(json)) hasData = true;
@@ -141,7 +174,7 @@ export function useDataPolling() {
'name' in e && 'name' in e &&
(e as { name?: string }).name === 'AbortError'; (e as { name?: string }).name === 'AbortError';
if (!aborted) { if (!aborted) {
console.error("Failed fetching fast live data", e); console.warn("Fast live data fetch will retry after runtime is reachable", e);
setStoreBackendStatus('disconnected'); setStoreBackendStatus('disconnected');
} }
} finally { } finally {
@@ -177,7 +210,7 @@ export function useDataPolling() {
'name' in e && 'name' in e &&
(e as { name?: string }).name === 'AbortError'; (e as { name?: string }).name === 'AbortError';
if (!aborted) { if (!aborted) {
console.error("Failed fetching slow live data", e); console.warn("Slow live data fetch will retry after runtime is reachable", e);
} }
} finally { } finally {
if (slowAbortRef.current === controller) { if (slowAbortRef.current === controller) {
@@ -191,7 +224,8 @@ export function useDataPolling() {
const scheduleNext = (tier: 'fast' | 'slow') => { const scheduleNext = (tier: 'fast' | 'slow') => {
if (tier === 'fast') { if (tier === 'fast') {
const delay = hasData ? 15000 : 3000; // 3s startup retry → 15s steady state const delay = hasData ? 15000 : 3000; // 3s startup retry → 15s steady state
fastTimerId = setTimeout(fetchFastData, delay); const needsFullFastPayload = fetchedStartupFastPayload && !fastEtag.current;
fastTimerId = setTimeout(fetchFastData, needsFullFastPayload ? 750 : delay);
} else { } else {
const delay = hasData ? 120000 : 5000; // 5s startup retry → 120s steady state const delay = hasData ? 120000 : 5000; // 5s startup retry → 120s steady state
slowTimerId = setTimeout(fetchSlowData, delay); slowTimerId = setTimeout(fetchSlowData, delay);
@@ -208,8 +242,12 @@ export function useDataPolling() {
}; };
window.addEventListener(LAYER_TOGGLE_EVENT, onLayerToggle); window.addEventListener(LAYER_TOGGLE_EVENT, onLayerToggle);
fetchFastData(); void (async () => {
fetchSlowData(); await fetchCriticalBootstrap();
fetchFastData();
// Let the bootstrap/fast payload paint before competing with the slow tier.
slowTimerId = setTimeout(fetchSlowData, 5000);
})();
return () => { return () => {
window.removeEventListener(LAYER_TOGGLE_EVENT, onLayerToggle); window.removeEventListener(LAYER_TOGGLE_EVENT, onLayerToggle);
+2 -2
View File
@@ -387,7 +387,7 @@ export async function refreshSnapshotList(): Promise<void> {
const json = await res.json(); const json = await res.json();
updateTimelineFromSnapshots(sortSnapshots(json.snapshots || [])); updateTimelineFromSnapshots(sortSnapshots(json.snapshots || []));
} catch (e) { } catch (e) {
console.error('Time Machine: failed to fetch snapshots', e); console.warn('Time Machine snapshots will retry after runtime is reachable', e);
} }
} }
@@ -402,7 +402,7 @@ export async function refreshHourlyIndex(): Promise<void> {
setState({ hourlyIndex: json.hours || {} }); setState({ hourlyIndex: json.hours || {} });
} }
} catch (e) { } catch (e) {
console.error('Time Machine: failed to fetch hourly index', e); console.warn('Time Machine hourly index will retry after runtime is reachable', e);
} }
} }
+16 -2
View File
@@ -21,6 +21,7 @@ export interface InfonetBootstrapSnapshot {
sync_peer_count?: number; sync_peer_count?: number;
push_peer_count?: number; push_peer_count?: number;
operator_peer_count?: number; operator_peer_count?: number;
bootstrap_seed_peer_count?: number;
default_sync_peer_count?: number; default_sync_peer_count?: number;
last_bootstrap_error?: string; last_bootstrap_error?: string;
} }
@@ -69,6 +70,7 @@ export interface InfonetNodeStatusSnapshot {
sync_runtime?: InfonetSyncRuntimeSnapshot; sync_runtime?: InfonetSyncRuntimeSnapshot;
push_runtime?: InfonetPushRuntimeSnapshot; push_runtime?: InfonetPushRuntimeSnapshot;
private_lane_tier?: string; private_lane_tier?: string;
private_transport_required?: boolean;
} }
export interface NodeSettingsSnapshot { export interface NodeSettingsSnapshot {
@@ -79,9 +81,14 @@ export interface NodeSettingsSnapshot {
node_enabled?: boolean; node_enabled?: boolean;
} }
export const DEFAULT_INFONET_SEED_URL = 'https://node.shadowbroker.info'; export interface TorHiddenServiceSnapshot {
ok?: boolean;
running?: boolean;
onion_address?: string;
detail?: string;
}
const CACHE_TTL_MS = 15000; const CACHE_TTL_MS = 5000;
type CacheEntry<T> = { type CacheEntry<T> = {
value: T; value: T;
@@ -220,3 +227,10 @@ export async function setInfonetNodeEnabled(enabled: boolean): Promise<NodeSetti
invalidateInfonetNodeStatusCache(); invalidateInfonetNodeStatusCache();
return result; return result;
} }
export async function startTorHiddenService(): Promise<TorHiddenServiceSnapshot> {
return controlPlaneJson<TorHiddenServiceSnapshot>('/api/settings/tor/start', {
method: 'POST',
requireAdminSession: false,
});
}
+7 -4
View File
@@ -8,13 +8,13 @@
import { NextRequest, NextResponse } from 'next/server'; import { NextRequest, NextResponse } from 'next/server';
function buildCsp(nonce: string): string { function buildCsp(_nonce: string): string {
const isDev = process.env.NODE_ENV !== 'production'; const isDev = process.env.NODE_ENV !== 'production';
const directives = [ const directives = [
"default-src 'self'", "default-src 'self'",
isDev isDev
? `script-src 'self' 'unsafe-inline' 'unsafe-eval' 'nonce-${nonce}' blob:` ? "script-src 'self' 'unsafe-inline' 'unsafe-eval' blob:"
: `script-src 'self' 'nonce-${nonce}' blob:`, : "script-src 'self' 'unsafe-inline' blob:",
"style-src 'self' 'unsafe-inline' https://fonts.googleapis.com", "style-src 'self' 'unsafe-inline' https://fonts.googleapis.com",
"img-src 'self' data: blob: https:", "img-src 'self' data: blob: https:",
isDev isDev
@@ -35,7 +35,10 @@ function buildCsp(nonce: string): string {
export function middleware(request: NextRequest) { export function middleware(request: NextRequest) {
const nonce = Buffer.from(crypto.randomUUID()).toString('base64'); const nonce = Buffer.from(crypto.randomUUID()).toString('base64');
// Forward nonce to server components via request header. // Forward a nonce for future fully-wired CSP support. Do not include it in
// script-src until every Next inline bootstrap script receives the nonce;
// otherwise production hydration can fail and leave the app on the static
// "prioritizing map feeds" shell.
const requestHeaders = new Headers(request.headers); const requestHeaders = new Headers(request.headers);
requestHeaders.set('x-nonce', nonce); requestHeaders.set('x-nonce', nonce);
+5
View File
@@ -114,6 +114,7 @@ export interface Ship {
source_url?: string; source_url?: string;
last_osint_update?: string; last_osint_update?: string;
desc?: string; desc?: string;
trail?: Array<{ lat: number; lng: number; sog?: number; ts?: number } | number[]>;
// Tracked yacht enrichment // Tracked yacht enrichment
yacht_alert?: boolean; yacht_alert?: boolean;
yacht_owner?: string; yacht_owner?: string;
@@ -635,6 +636,8 @@ export interface NewsArticle {
kalshi_pct: number | null; kalshi_pct: number | null;
consensus_pct: number | null; consensus_pct: number | null;
match_score: number; match_score: number;
slug?: string;
kalshi_ticker?: string;
} | null; } | null;
} }
@@ -824,6 +827,8 @@ export interface DashboardData {
cctv_total?: number; cctv_total?: number;
satnogs_total?: number; satnogs_total?: number;
tinygs_total?: number; tinygs_total?: number;
bootstrap_ready?: boolean;
bootstrap_payload?: boolean;
sigint_totals?: { sigint_totals?: {
total?: number; total?: number;
meshtastic?: number; meshtastic?: number;
+2 -1
View File
@@ -1,7 +1,8 @@
--- ---
apiVersion: v2 apiVersion: v2
name: shadowbroker name: shadowbroker
version: 0.0.1 version: 0.9.75
appVersion: "0.9.75"
description: simple shadowbroker installation description: simple shadowbroker installation
type: application type: application
+1 -1
View File
@@ -1,6 +1,6 @@
[project] [project]
name = "shadowbroker" name = "shadowbroker"
version = "0.9.7" version = "0.9.75"
readme = "README.md" readme = "README.md"
requires-python = ">=3.10" requires-python = ">=3.10"
dependencies = [] dependencies = []
+100
View File
@@ -0,0 +1,100 @@
from __future__ import annotations
import hashlib
import re
from pathlib import Path
TRUE_VALUES = {"1", "true", "yes", "on", "allow", "enabled"}
PIN_KEY = "PRIVACY_CORE_ALLOWED_SHA256"
PRIVATE_LANE_KEYS = ("MESH_ARTI_ENABLED", "MESH_RNS_ENABLED")
def _repo_root() -> Path:
return Path(__file__).resolve().parents[1]
def _privacy_core_library(root: Path) -> Path | None:
release_dir = root / "privacy-core" / "target" / "release"
candidates = (
release_dir / "privacy_core.dll",
release_dir / "libprivacy_core.so",
release_dir / "libprivacy_core.dylib",
)
for candidate in candidates:
if candidate.is_file():
return candidate
return None
def _parse_env(lines: list[str]) -> dict[str, str]:
values: dict[str, str] = {}
for line in lines:
match = re.match(r"^\s*([A-Za-z_][A-Za-z0-9_]*)\s*=\s*(.*)\s*$", line)
if not match:
continue
key, raw_value = match.groups()
values[key] = raw_value.strip().strip('"').strip("'")
return values
def _private_lane_enabled(values: dict[str, str]) -> bool:
for key in PRIVATE_LANE_KEYS:
value = values.get(key, "")
if value.strip().lower() in TRUE_VALUES:
return True
return False
def _replace_or_append_pin(lines: list[str], digest: str) -> tuple[list[str], bool]:
updated: list[str] = []
replaced = False
pattern = re.compile(rf"^(\s*{re.escape(PIN_KEY)}\s*=).*$")
for line in lines:
if pattern.match(line):
updated.append(f"{PIN_KEY}={digest}")
replaced = True
else:
updated.append(line)
if not replaced:
if updated and updated[-1].strip():
updated.append("")
updated.append(f"{PIN_KEY}={digest}")
return updated, replaced
def main() -> int:
root = _repo_root()
env_path = root / "backend" / ".env"
if not env_path.is_file():
print("[*] privacy-core trust pin refresh skipped: backend/.env not found.")
return 0
library_path = _privacy_core_library(root)
if library_path is None:
print("[*] privacy-core trust pin refresh skipped: shared library not found.")
return 0
text = env_path.read_text(encoding="utf-8-sig")
lines = text.splitlines()
values = _parse_env(lines)
has_pin = PIN_KEY in values
if not has_pin and not _private_lane_enabled(values):
print("[*] privacy-core trust pin refresh skipped: private-lane mode is not enabled.")
return 0
digest = hashlib.sha256(library_path.read_bytes()).hexdigest()
if values.get(PIN_KEY, "").strip().lower() == digest:
print("[*] privacy-core trust pin already current.")
return 0
updated, replaced = _replace_or_append_pin(lines, digest)
newline = "\r\n" if "\r\n" in text else "\n"
env_path.write_text(newline.join(updated) + newline, encoding="utf-8")
action = "refreshed" if replaced else "enrolled"
print(f"[*] privacy-core trust pin {action} for local shared library.")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+31
View File
@@ -0,0 +1,31 @@
param(
[string]$Root = (Resolve-Path (Join-Path $PSScriptRoot "..")).Path
)
$ErrorActionPreference = "Stop"
$Host.UI.RawUI.WindowTitle = "ShadowBroker Runtime"
Set-Location -LiteralPath $Root
Write-Host "==================================================="
Write-Host " ShadowBroker runtime"
Write-Host " Dashboard: http://localhost:3000"
Write-Host " Close this window or press Ctrl+C to stop."
Write-Host "==================================================="
Write-Host ""
try {
& node "frontend\scripts\dev-all.cjs"
$exitCode = $LASTEXITCODE
} catch {
Write-Host ""
Write-Host "[!] Runtime failed: $($_.Exception.Message)"
$exitCode = 1
}
Write-Host ""
Write-Host "==================================================="
Write-Host " ShadowBroker has stopped. Exit code: $exitCode"
Write-Host "==================================================="
Read-Host "Press Enter to close"
exit $exitCode
+44 -1
View File
@@ -84,6 +84,9 @@ for /f "tokens=5" %%a in ('netstat -ano ^| findstr ":8000 "') do (
for /f "tokens=5" %%a in ('netstat -ano ^| findstr ":3000 "') do ( for /f "tokens=5" %%a in ('netstat -ano ^| findstr ":3000 "') do (
taskkill /F /PID %%a >nul 2>&1 taskkill /F /PID %%a >nul 2>&1
) )
for /f "tokens=5" %%a in ('netstat -ano ^| findstr ":8787 "') do (
taskkill /F /PID %%a >nul 2>&1
)
:: Brief pause to let OS release the ports :: Brief pause to let OS release the ports
timeout /t 1 /nobreak >nul timeout /t 1 /nobreak >nul
@@ -99,6 +102,11 @@ if %errorlevel% equ 0 (
echo [!] WARNING: Port 3000 is still occupied! Waiting 3s for OS cleanup... echo [!] WARNING: Port 3000 is still occupied! Waiting 3s for OS cleanup...
timeout /t 3 /nobreak >nul timeout /t 3 /nobreak >nul
) )
netstat -ano | findstr ":8787 " | findstr "LISTENING" >nul 2>&1
if %errorlevel% equ 0 (
echo [!] WARNING: Port 8787 is still occupied! Waiting 3s for OS cleanup...
timeout /t 3 /nobreak >nul
)
echo [*] Ports clear. echo [*] Ports clear.
:: ──────────────────────────────────────────────────────────────────── :: ────────────────────────────────────────────────────────────────────
@@ -225,6 +233,40 @@ if not exist "node_modules\ws" (
call npm ci --omit=dev --silent call npm ci --omit=dev --silent
) )
echo [*] Backend Node.js dependencies OK. echo [*] Backend Node.js dependencies OK.
echo.
echo [*] Checking privacy-core shared library...
set "PRIVACY_CORE_DLL=%ROOT%\privacy-core\target\release\privacy_core.dll"
if not exist "%PRIVACY_CORE_DLL%" (
where cargo >nul 2>&1
if errorlevel 1 (
echo [!] WARNING: privacy-core DLL is missing and Rust/Cargo is not installed.
echo [!] Infonet private lanes and gates need this library.
echo [!] Install Rust from https://rustup.rs/ and run:
echo [!] cargo build --release --manifest-path "%ROOT%\privacy-core\Cargo.toml"
echo.
) else (
echo [*] Building privacy-core release DLL...
cd /d "%ROOT%"
cargo build --release --manifest-path "%ROOT%\privacy-core\Cargo.toml"
if errorlevel 1 (
echo [!] ERROR: privacy-core build failed. Infonet private lanes need this DLL.
echo.
pause
exit /b 1
)
cd /d "%ROOT%\backend"
)
)
if exist "%PRIVACY_CORE_DLL%" (
echo [*] privacy-core DLL OK.
"%VENV_PY%" "%ROOT%\scripts\refresh_privacy_core_pin.py"
if errorlevel 1 (
echo [!] WARNING: privacy-core trust pin refresh failed. Startup may fail if backend\.env pins an old hash.
echo.
)
)
cd /d "%ROOT%" cd /d "%ROOT%"
echo. echo.
@@ -257,7 +299,8 @@ echo ===================================================
echo (Press Ctrl+C to stop) echo (Press Ctrl+C to stop)
echo. echo.
call npm run dev start "ShadowBroker Runtime" powershell.exe -NoProfile -ExecutionPolicy Bypass -NoExit -File "%ROOT%\scripts\run-windows-runtime.ps1" -Root "%ROOT%"
exit /b 0
echo. echo.
echo =================================================== echo ===================================================

Some files were not shown because too many files have changed in this diff Show More