mirror of
https://github.com/BigBodyCobain/Shadowbroker.git
synced 2026-08-27 21:00:28 +02:00
[security] Close post-#227 control-surface and fetcher gaps
PR #227 hardened most Wormhole/Infonet control surfaces behind require_local_operator and made the CrowdThreat fetcher opt-in. An audit of the codebase against that PR's stated goals turned up four classes of gap that the original change missed: 1. Two operator-only endpoints were left unprotected: - POST /api/wormhole/join: calls bootstrap_wormhole_identity() and flips the node into Tor mode, exactly the surface #227 hardened on /api/wormhole/identity/bootstrap. - POST /api/sigint/transmit: relays APRS-IS packets over radio using operator-supplied credentials. Anything that reached the API could transmit on the operator's authority. Both now require_local_operator. test_control_surface_auth.py extended with regression coverage for both. 2. Five third-party fetchers were still default-on, phoning home to politically/commercially sensitive upstreams on every poll cycle: - fimi.py -> euvsdisinfo.eu -> FIMI_ENABLED - prediction_markets -> Polymarket + Kalshi -> PREDICTION_MARKETS_ENABLED - financial.py -> Finnhub / yfinance -> FINANCIAL_ENABLED or FINNHUB_API_KEY - nuforc_enrichment -> huggingface.co -> NUFORC_ENABLED - news.py -> configured RSS feeds -> NEWS_ENABLED (default on, kill switch) Same CrowdThreat-style pattern: explicit env-var opt-in, empty the data slot and mark_fresh when disabled. New regression test file test_third_party_fetchers_opt_in.py asserts each fetcher's network entry point is not called when its gate is off. 3. The outbound User-Agent leaked both the operator's personal email and a fork-specific GitHub URL on every fetcher request. Consolidated to a single DEFAULT_USER_AGENT in network_utils.py, project-generic by default (no contact info), overridable via SHADOWBROKER_USER_AGENT for operators who want to identify themselves (e.g. for Nominatim or weather.gov usage-policy compliance). Six call sites updated; the Nominatim-specific override is preserved. 4. The same generic UA now also flows through the peer prekey lookup in mesh_wormhole_prekey.py, so DM first-contact requests no longer identify the caller as a Shadowbroker fork to the peer being queried. .env.example updated to document all new opt-in env vars. Tests: backend/tests/test_control_surface_auth.py (extended), backend/tests/test_crowdthreat_opt_in.py (unchanged, still passes), backend/tests/test_third_party_fetchers_opt_in.py (new, 7 tests). All 31 tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.7
parent
de27d119f9
commit
71a9d9e144
@@ -5,6 +5,7 @@ debunked claims, threat actor mentions, and target country references.
|
||||
Refreshes every 12 hours (FIMI data updates weekly).
|
||||
"""
|
||||
|
||||
import os
|
||||
import re
|
||||
import logging
|
||||
from datetime import datetime, timezone
|
||||
@@ -18,6 +19,16 @@ logger = logging.getLogger("services.data_fetcher")
|
||||
|
||||
_FIMI_FEED_URL = "https://euvsdisinfo.eu/feed/"
|
||||
|
||||
|
||||
def fimi_fetch_enabled() -> bool:
|
||||
"""Return True only when the operator explicitly opts into FIMI pulls."""
|
||||
return str(os.environ.get("FIMI_ENABLED", "")).strip().lower() in {
|
||||
"1",
|
||||
"true",
|
||||
"yes",
|
||||
"on",
|
||||
}
|
||||
|
||||
# ── Threat actor keywords ──────────────────────────────────────────────────
|
||||
# Map of keyword → canonical actor name. Checked case-insensitively.
|
||||
_THREAT_ACTORS: dict[str, str] = {
|
||||
@@ -173,6 +184,12 @@ def _is_major_wave(narratives: list[dict], targets: dict[str, int]) -> bool:
|
||||
@with_retry(max_retries=1, base_delay=5)
|
||||
def fetch_fimi():
|
||||
"""Fetch and parse the EUvsDisinfo RSS feed."""
|
||||
if not fimi_fetch_enabled():
|
||||
logger.debug("FIMI fetch skipped; set FIMI_ENABLED=true to opt in")
|
||||
with _data_lock:
|
||||
latest_data["fimi"] = []
|
||||
_mark_fresh("fimi")
|
||||
return
|
||||
try:
|
||||
resp = fetch_with_curl(_FIMI_FEED_URL, timeout=15)
|
||||
feed = feedparser.parse(resp.text)
|
||||
|
||||
Reference in New Issue
Block a user