* fix(gdelt): publish enriched copies instead of mutating live features
_enrich_gdelt_titles_background ran in a daemon thread that mutated the
nested properties dicts of GDELT features already published into
latest_data[gdelt]. HTTP readers hold live references to those dicts and
serialize them outside the data lock, so the in-place mutation raced the
serializer and raised RuntimeError: dictionary changed size during
iteration on /api/live-data/slow and /api/bootstrap/critical.
Enrich deep copies instead and atomically swap the top-level key under
_data_lock, with an identity guard so a newer fetch_gdelt() is not clobbered.
Honors the replace-don't-mutate contract documented in fetchers/_store.py.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(gdelt): regression test for background enrichment isolation
Asserts _enrich_gdelt_titles_background does not mutate already-published features and instead atomically swaps latest_data["gdelt"] with enriched copies (with the identity guard). Locks in the fix for the dictionary-changed-size race.
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
get_latest_data_deepcopy_snapshot deep-copies layers outside the data lock; a writer mutating a nested object in place races it and raises "dictionary changed size during iteration" (500 on /api/health, /api/live-data). Two changes: (1) _merge_sigint_snapshot now shallow-copies each entry so latest_data["sigint"] no longer aliases the SIGINT bridge dicts or the meshtastic_map_nodes layer (the concrete offender); (2) the snapshot retries a few times as defense-in-depth for any other in-place mutator. Plus regression tests.
Unify toggle-all exclusions for Earth imagery overlays so the icon matches layer state, and let Docker operators save news feeds via the proxy without a misleading network error.
Dashboard VPS nodes can set MESH_INFONET_FLEET_JOIN=false to avoid Tor
manifest sync wedging the API during OSINT warmup.
Co-authored-by: Cursor <cursoragent@cursor.com>
E2E harness recreates the full dashboard backend when a run ends so local
map layers are not left in lean MESH_ONLY mode. Participant compose no
longer forces MESH_ONLY=true — set it in .env only for lean DM-only nodes.
Co-authored-by: Cursor <cursoragent@cursor.com>
Step 8 uses live HTTP poll/decrypt instead of wedging remote python;
prime local wormhole before Tor warmup; auto-set MESH_RELAY_PEERS on
participant prime. Verified Extra run 119 and Pete Tor-only run 121.
Co-authored-by: Cursor <cursoragent@cursor.com>
Adds connect-contact HTTP endpoint with cached-bundle support, subprocess contact send via docker cp bundle file, and direct Tor prekey fetch to avoid wedging single-worker uvicorn.
Co-authored-by: Cursor <cursoragent@cursor.com>
Avoids wedging single-worker local uvicorn on long /dm/pubkey aggregator calls when testing new fleet onions like vps-extra.
Co-authored-by: Cursor <cursoragent@cursor.com>
Supports third-party participants deployed via compose pull; includes wormhole prime helper for fresh VPS nodes.
Co-authored-by: Cursor <cursoragent@cursor.com>
REMOTE_PARTICIPANT_ONION aliases PETE_ONION so the same script can target a non-Pete peer once deployed.
Co-authored-by: Cursor <cursoragent@cursor.com>
MLS export/reset and accept use live HTTP so uvicorn privacy-core state stays consistent; relay persistence and sender_seal fixes enable invite-accept-shared decrypt across onion peers. Adds participant/e2e compose overlays and harness recovery with optional Tor-only replicate mode.
Co-authored-by: Cursor <cursoragent@cursor.com>
Retry announce/manifest while Tor circuits warm on NODE and startup bootstrap.
Add verify_swarm_fresh_participant.py for empty-volume GHCR smoke tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
fetchDmPublicKey now passes allowLegacyAgentId: false for short-address
contact requests; update the assertion to match the new call signature.
Co-authored-by: Cursor <cursoragent@cursor.com>
Auto-relay connect DMs with End Contact severing, signed fleet prekey lookup,
OpenClaw private Infonet channel intents, headless relay Tor bootstrap on redeploy,
and swarm/DM live verification scripts.
Co-authored-by: Cursor <cursoragent@cursor.com>
Seed relay nodes should track CI-published backend images instead of local builds that fail without full monorepo context.
Co-authored-by: Cursor <cursoragent@cursor.com>
Ship sb-testnet fleet defaults, swarm/join API, NODE launcher registration step, and meshnode script defaults so users discover peers via the signed seed manifest without manual peer lists.
Co-authored-by: Cursor <cursoragent@cursor.com>
Signed peer manifest pull/announce on the seed, immediate hashchain push for gate messages, seed-only Docker defaults, and stale-genesis sync diagnostics.
Co-authored-by: Cursor <cursoragent@cursor.com>
Update gate-resync decomposition expectations for Infonet embed and harden GateView stream snapshot waits for slower CI runners.
Co-authored-by: Cursor <cursoragent@cursor.com>
Rename Mesh Chat to Meshtastic Chat, embed the Infonet terminal with Arti/Tor warmup, improve the agent shell PTY (git in the backend image, operator PATH), and add docker-compose.override for local image builds. Gitignore Hermes Agent runtime installs.
Co-authored-by: Cursor <cursoragent@cursor.com>
Uses a local-operator WebSocket bash session, keeps the map interactive, and SNAP docks the shell back into Mesh Chat instead of a floating blurred panel.
Co-authored-by: Cursor <cursoragent@cursor.com>
Anchors to the Mesh Chat box, stretches on tab enter, and supports user resize without changing the fixed left column width.
Co-authored-by: Cursor <cursoragent@cursor.com>
Carry SHA-256 hashes for the source zip, MSI, and setup EXE into release_digests.json while retaining prior release entries.
Co-authored-by: Cursor <cursoragent@cursor.com>
Align backend, desktop, helm, and frontend package versions for the Telegram OSINT and OpenClaw recon release.
Co-authored-by: Cursor <cursoragent@cursor.com>
Add Telegram OSINT with hourly incremental t.me scraping, metro geocoding
separate from news centroids, threat-intercept popup UI with inline media,
and HTML markers above alert boxes so pins stay clickable. Expose GFW_API_TOKEN
in onboarding and Settings Maritime; harden GFW/CCTV/geo fetchers. Port Osiris-
derived recon, SCM, entity graph, malware/cyber feeds, sanctions, and submarine
cable layers with tests and documentation.
Co-authored-by: Cursor <cursoragent@cursor.com>
Align full /api/live-data with slow-tier orjson options, remove dead main.py duplicate, cap slow batches to pool size, cancel queued work on timeout, and stop retrying HTTP 4xx/5xx.
Co-authored-by: Cursor <cursoragent@cursor.com>
Default python main.py to loopback, deep-copy dashboard snapshots outside the store lock with ETag on full live-data, and route GDELT/LiveUAMap/CCTV/slow-tier work through an isolated executor so Playwright jobs cannot starve fast-tier workers.
Co-authored-by: Cursor <cursoragent@cursor.com>
Operators enable Polymarket/Kalshi correlation from Global Threat Intercept with a consent dialog; polls use a jittered schedule separate from the slow tier. Right-click Sentinel imagery returns up to three signed scenes again.
Co-authored-by: Cursor <cursoragent@cursor.com>
Each install pulls ~60-day sightings from nuforc.org every Monday; disk cache
matches weekly cadence so users keep current pins between restarts.
Co-authored-by: Cursor <cursoragent@cursor.com>
Filter stale rows out of nuforc_recent_sightings.json on load; add requests-based
live scrape when curl is disabled; daily scheduler rebuild instead of weekly-only.
Co-authored-by: Cursor <cursoragent@cursor.com>
- User-Agent is per-install handle only (no Shadowbroker product token)
- LiveUAMap: Windows UI consent when enabling Global Incidents; env override
- Meshtastic callsign upstream header off by default (opt-in true)
- Expanded docs/OUTBOUND_DATA.md and README link for CCTV, basemap, Broadcastify
Co-authored-by: Cursor <cursoragent@cursor.com>
Operators can set DEEPSTATE_MIRROR_COMMIT for immutable frontline ingest; Madrid KML tries HTTPS then HTTP without changing camera image URLs or proxy Referers.
Co-authored-by: Cursor <cursoragent@cursor.com>
Shadowbroker