The repo root has been accumulating AI-coding-agent dropouts that have
no project contract value:
.codex/, .codex-app-schema/, .codex-app-ts/ — OpenAI Codex CLI
AGENTS.md, GEMINI.md — per-agent instructions
CLAUDE.md — same shape
.github/copilot-instructions.md — GitHub Copilot hints
These are operator-side preferences. If something needs to be canonical
for the project, it goes in docs/ explicitly.
Also adding backend/tests/test_carrier_tracker_region_centers.py —
a stale fixture that referenced fields (region, source_detail,
position_label, position_source_type, position_confidence='low')
that don't exist in the current `_parse_carrier_positions_from_news`
implementation. The real coverage for that function lives in
tests/test_carrier_tracker_quality.py from PR #285.
Replace the dated editorial fallback positions baked into the registry
with a one-shot seed file + persistent observation cache. The user's
runtime cache now reflects what THIS install has actually observed,
not what USNI published on March 9, 2026. A year from now, the cache
holds a year of observations and the seed is irrelevant.
== #244: dated editorial coordinates out of the registry ==
CARRIER_REGISTRY no longer carries fallback_lat/lng/heading/desc.
Those fields are deleted. The registry is now identity + homeport
only.
New file: backend/data/carrier_seed.json
- Read-only, shipped with every release.
- Used ONCE on first-ever startup to bootstrap carrier_cache.json.
- Each entry stamped with position_confidence="seed" and the actual
as-of date (2026-03-09), NOT now().
== #245: approximate confidence for headline-derived positions ==
_parse_carrier_positions_from_news() now stamps every GDELT-derived
entry with position_confidence="approximate" so the UI knows the
coordinate is a region-centroid match, not a precise observation.
After the freshness window the label rolls over to
"stale_approximate" so old-and-imprecise is distinguishable from
recent-and-imprecise.
The article's actual seendate is used as position_source_at instead
of now(), so the "last reported X days ago" badge is honest.
== #246: freshness is labelling, not eviction ==
The cache always preserves the last position the system observed,
forever. What changes is the position_confidence label:
- within configurable window (default 14d, env-overridable via
SHADOWBROKER_CARRIER_FRESHNESS_DAYS) -> "recent"
- older -> "stale"
- seed-bootstrap entries that were never refreshed -> "seed"
- homeport defaults (carrier added post-install) -> "homeport_default"
- headline-derived (any age, fresh) -> "approximate"
- headline-derived (older than window) -> "stale_approximate"
The position itself never reverts to the seed or the registry. The
user always sees the last position the system observed. Per the
user's explicit guidance: "from there have it be the last position
the user has logged the carriers that way a year from now it doesnt
revert to where the ships are today".
== Other improvements ==
- CACHE_FILE moved to backend/data/carrier_cache.json so it lives in
the volume-mounted dir under Docker compose. Previously it was at
/app/carrier_cache.json which got wiped on every container restart
(pre-existing bug).
- Atomic cache write (temp + os.replace) so a crash mid-write does
not leave a truncated cache file.
== Public API shape ==
Every carrier object the API emits now includes:
- position_confidence: seed | recent | stale | approximate |
stale_approximate | homeport_default
- position_source_at: ISO timestamp of when the underlying source
was observed (NOT now())
- is_fallback: convenience boolean for the UI; true when the
confidence is seed/stale/stale_approximate/
homeport_default
Existing fields (estimated, source, source_url, last_osint_update,
name, type, lat, lng, country, desc, wiki) are preserved exactly so
the current ShipPopup frontend renders unchanged. last_osint_update
now reflects position_source_at instead of now(), which is what the
existing "last reported MM/DD" badge always meant to show.
Tests: backend/tests/test_carrier_tracker_quality.py — 17 tests
covering seed bootstrap, subsequent-startup ignoring seed, no-seed/
no-cache homeport fallback, registry no longer has fallback fields,
freshness window labelling + env override, "year-old cache entry keeps
its position, only the label flips" regression, approximate
confidence for headline matches, GDELT seendate ISO parser, public
response shape backward compat.
Credit: tg12 (external security audit, three P1/P2 issues).
External audit (@tg12, May 18) found that backend/services/updater.py
silently skipped all SHA-256 integrity verification whenever the
MESH_UPDATE_SHA256 env var was unset — which is the default. Nothing
in any install doc tells operators to set it, so practically every
deployment was running the auto-updater with zero integrity check.
That made GitHub release pipeline compromise a single-step path to
arbitrary code execution on every node that auto-updates.
Investigation surfaced a deeper bug too: the updater downloads
zipball_url (GitHub's auto-generated source archive) but the
maintainer's release process publishes SHA256SUMS.txt for a separate
named asset (ShadowBroker_v*.zip). So even if MESH_UPDATE_SHA256
WERE set, operators had no published digest to compare against — the
file they were downloading wasn't the file the maintainer had signed.
This PR fixes both issues with the same multi-source verification
chain we shipped for the Tor bundle in PR #261:
backend/services/updater.py
_download_release() now prefers a maintainer-signed release asset
matching ShadowBroker_v*.zip over zipball_url. Captures the
SHA256SUMS.txt asset URL when present.
_validate_zip_hash() rewritten as a four-source chain:
1. MESH_UPDATE_SHA256 env var (operator override, preserved)
2. SHA256SUMS.txt asset published with the release (primary —
the maintainer's release process already publishes this)
3. Baked-in backend/data/release_digests.json (second line of
defense for releases that lack the SHA256SUMS asset, or when
the asset can't be fetched at update time)
4. HTTPS-only fallback with a loud warning (preserves the auto-
update flow during transient outages)
Mismatch from any source that DID respond is fatal — the update
is refused and the existing install keeps running. Only the
"no source reachable at all" case falls back to HTTPS-only.
_fetch_sha256sums() new — fetches and parses a standard
SHA256SUMS.txt asset. Handles both "<digest> <name>" and binary-
marker "<digest> *<name>" formats. Tolerant to comments, blank
lines, and malformed entries.
backend/data/release_digests.json (new)
Baked-in digest list keyed by release tag. Seeded with the v0.9.79
entries copied from the published SHA256SUMS.txt:
ShadowBroker_v0.9.79.zip = f6877c1d6661...
ShadowBroker_0.9.79_x64-setup.exe = f7b676ada45c...
ShadowBroker_0.9.79_x64_en-US.msi = e0713c3cdda1...
Whitelisted in .gitignore alongside the other static reference
data files (kiwisdr_directory.json, tor_bundle_digests.json,
aisstream_spki_pins.json).
backend/tests/test_update_integrity_chain.py (new, 16 tests)
- Each source matches → success, identifies which source verified
- Each source mismatches → RuntimeError "mismatch"
- No source reachable → https-only fallback with loud warning
- Env override beats all other sources (preserved precedence)
- SHA256SUMS.txt parser handles standard, binary-marker, comments,
and network-failure cases
Validation:
pytest backend/tests/test_update_integrity_chain.py → 16 passed
pytest (all 15 security test files together) → 105 passed
UX impact: zero. Normal auto-update flow is unchanged for legitimate
releases (path 2 catches everything because the release publishes
SHA256SUMS.txt). Transient network failures during update gracefully
fall through to path 3 then path 4 — no operator intervention needed.
The only user-visible behavior change is in the compromised-release
case, where the update is now refused instead of silently applied.
Credit: @tg12 for the original bug report and the specific call-out
that MESH_UPDATE_SHA256 was unreachable by default operators.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
External report from @jmleclercq: AISStream's Let's Encrypt cert
expired on 2026-05-20 (verified — their renewal pipeline failed), so
the AIS WebSocket connection dies with CERT_HAS_EXPIRED and the
maritime layer empties out. The reporter worked around it locally by
passing { rejectUnauthorized: false } to the WebSocket constructor and
asked whether we should add an env var for that.
That fix is the wrong fix. Disabling TLS validation entirely lets any
network attacker MITM the WebSocket and inject fake ship positions —
same class as the GDELT plaintext-HTTP MITM we just closed in #199.
Adding an env var for it would be an attractive nuisance: operators
set it once during a bad cert week and then forget, leaving themselves
open to MITM forever.
Right fix: SPKI pinning, same pattern as the Tor bundle digest pinning
in #201. The insight is that Let's Encrypt renewals keep the SAME
public key by default, so the SPKI hash survives normal cert rotation.
We can relax the date check while keeping the identity check.
Mechanics:
backend/data/aisstream_spki_pins.json (new)
Pinned SHA-256 hashes of the DER-encoded SPKI bytes for
stream.aisstream.io. Captured 2026-05-20 from the live cert.
Format is base64(sha256(pubkey_der)), matching the canonical
openssl pipeline. Whitelisted in .gitignore alongside the other
static reference data files (KiwiSDR directory, Tor bundle
digests).
backend/ais_proxy.js
Path A (99.9% of the time): normal TLS validation. Untouched.
Path B (on CERT_HAS_EXPIRED only): re-handshake with
rejectUnauthorized=false JUST to read the leaf cert, compute its
SPKI hash, compare against the pinned list. If match → upstream
is still the genuine AISStream → re-open the WebSocket with
rejectUnauthorized=false and log DEGRADED MODE. If no match →
refuse the connection, log loudly: this would be a real MITM.
Pin file is looked up in three locations so the same code works
in the Docker backend, the Tauri desktop runtime, and any
operator-relocated layout (SHADOWBROKER_AIS_PINS env var).
Embedded fallback list inside the JS so portable installs that
haven't shipped the JSON still work.
backend/services/ais_stream.py
Captures the proxy's status markers from stdout
({"__ais_proxy_status": {"degraded_tls": true}}) into a module-
level snapshot. Exposes ais_proxy_status() for the health
endpoint. Doesn't touch the data plane — degraded mode keeps
receiving vessel data, just with weaker MITM protection.
backend/routers/health.py + backend/services/schemas.py
/api/health now includes an ais_proxy block with degraded_tls.
Top-level status escalates ok -> degraded when AIS is in
degraded TLS mode (but won't downgrade a worse SLO status).
Operators get a visible signal that they're in degraded mode
without needing to grep logs.
Tests: backend/tests/test_ais_spki_pinning.py (7 tests)
- Pin file structure validation (JSON, host entry, base64 SHA-256)
- ais_proxy_status() snapshot semantics (starts empty, defensive copy)
- /api/health surfaces ais_proxy.degraded_tls when set
- /api/health returns empty ais_proxy when proxy hasn't reported
Node.js syntax check passes (node --check) on both backend/ais_proxy.js
and the Tauri runtime mirror.
When AISStream renews their cert (likely within hours-to-days), the
normal-TLS path succeeds on next reconnect and degraded_tls clears
automatically. No operator action needed. If they instead rotate their
server key, the SPKI check will fail and we'll need to add the new
hash to backend/data/aisstream_spki_pins.json before removing the old
one.
Credit: @jmleclercq for the clear report and the careful workaround
verification (Node version, ws version, manual probe).
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
External security audit by @tg12 (May 17, 2026) filed issues #201–#214
in addition to the #189–#200 batch already closed by PRs #227/#232/#260.
This PR closes all eight that are real security bugs (the other six in
the 201–214 range are either design discussions or upstream-abuse/TOS
concerns we're keeping intentional, see issue triage notes on each).
The user-facing principle for this PR: fix the security gap WITHOUT
introducing a single hostile error or behavior change for legitimate
users. Every fix follows the same template — fail forward, not loud.
When the secure path is harder than the insecure one, build a
fallback chain that ends in graceful degradation, not in a scary
modal or 422 response.
#205 — OpenMHZ audio redirect SSRF (services/radio_intercept.py)
Replaced requests.get(..., allow_redirects=True) with a manual
redirect loop that re-validates each hop's host against
_OPENMHZ_AUDIO_HOSTS. Same-host redirects (CDN edge selection)
still work, so legitimate audio playback is unaffected. Cross-host
redirects to disallowed hosts return a generic 502 which the
browser audio element handles gracefully. Cap at 5 hops.
#207 — infonet/status verify_signatures DoS (routers/mesh_public.py)
Silently downgrade verify_signatures=true to False for
unauthenticated callers. No error surfaced — the response shape is
identical, just without the O(n_events) signature verification.
Authenticated callers (scoped mesh.audit) still get the full path.
The frontend never passes this param so legitimate UI is unaffected.
#211 — thermal/verify expensive analysis (routers/sigint.py)
Added Depends(require_local_operator). Frontend has no direct
callers (verified by grep); Tauri/AI agents use scoped tokens that
pass the auth check. Anonymous abusers blocked silently — the
legitimate UI keeps working through the Next.js admin-key proxy.
#213, #214 — OpenMHZ calls/audio upstream abuse (routers/radio.py)
Added Depends(require_local_operator) to both. Browser users hit
these through the Next.js proxy at src/app/api/[...path]/route.ts
which injects X-Admin-Key, so the auth check passes transparently.
Direct attackers can no longer rotate sys_names to hammer
api.openmhz.com or relay arbitrary audio streams through the
backend's bandwidth.
#202 — overflights unbounded hours (routers/data.py)
Silently clamp `hours` to OVERFLIGHTS_MAX_HOURS (default 72,
configurable). NO 422 — clients asking for an absurd window get a
shorter window back with `requested_hours` and `effective_hours`
hint fields. Postel's law: liberal in what we accept, conservative
in what we compute.
#203 — Meshtastic callsign UA leak (services/fetchers/meshtastic_map.py)
Added MESHTASTIC_SEND_CALLSIGN_HEADER opt-out env var. Default is
TRUE — preserves existing operator behavior (callsign sent so
meshtastic.org can rate-limit per-install). Privacy-conscious
operators set it to false to suppress.
#206 — KiwiSDR upstream is HTTP-only (services/kiwisdr_fetcher.py)
Upstream rx.linkfanel.net doesn't speak HTTPS (verified — Apache
2.4.10 only on port 80). We can't fix the transport. Instead added
three layers:
1. Content validation on fetched data — reject responses with
<50 receivers or >5% malformed entries (likely MITM injection).
2. Existing disk cache fallback (already present).
3. NEW: bundled static directory at backend/data/kiwisdr_directory.json
shipping 798 known-good receivers. Used as last resort so the
KiwiSDR map layer always renders something useful.
#208 — Merkle proof DoS via /api/mesh/infonet/sync (services/mesh/mesh_hashchain.py)
The endpoint is part of the cross-node federation protocol — peers
legitimately call it without local-operator auth, so we can't add
Depends(). Instead made the underlying operation O(1) per proof
via a cached Merkle level structure on the Infonet instance:
- _merkle_levels_cache + _merkle_levels_for_event_count on each
Infonet instance
- _invalidate_merkle_cache() called from every chain mutation
point (append, ingest_events, apply_fork, cleanup_expired)
- _get_merkle_levels() does the lazy recompute on first read
after invalidation, then serves from cache thereafter
Effect: anonymous attackers hammering the proofs endpoint hit a
cached structure; the rebuild happens at most once per real chain
advance. Federation untouched.
#201 — Tor bundle SHA-256 bypass (services/tor_hidden_service.py)
Docker users were already covered — backend/Dockerfile installs
Tor via apt-get at build time (signed by Debian's package system).
No runtime download needed for the 80%-of-users case.
For Tauri desktop, replaced the single .sha256sum check with a
multi-source verification chain implemented in _verify_tor_bundle():
1. Try upstream .sha256sum (current behavior — fast path)
2. Try baked-in digest list at backend/data/tor_bundle_digests.json
(pinned per-version, maintainer-updated)
3. If neither source is REACHABLE: HTTPS-only fallback with a loud
warning (avoids breaking first-run onboarding while the
maintainer hasn't yet pinned a new Tor release)
A mismatch from a source that DID respond is always fatal — only
the "no source reachable" case falls back to HTTPS-only. This is
the "have cake and eat it" pattern: real users see no new failure
modes during torproject.org outages, but MITM/compromise attacks
still fail because the downloaded digest can't match what BOTH
the upstream and the baked-in list report.
Currently the digest file ships with placeholder values for the
current Tor URLs (those URLs are already stale on torproject.org
too). A follow-up commit can populate real digests when a stable
Tor release is selected; until then the HTTPS-only warning fires
and onboarding still works.
Tests (82 total, all passing):
test_openmhz_redirect_ssrf.py (5 tests) — #205
test_infonet_status_verify_gate.py (2 tests) — #207
test_overflights_clamp.py (5 tests) — #202
test_meshtastic_callsign_optout.py (3 tests) — #203
test_kiwisdr_fallback.py (6 tests) — #206
test_merkle_cache.py (6 tests) — #208
test_tor_bundle_verification.py (6 tests) — #201
test_control_surface_auth.py (extended) — #211, #213, #214
+ all previous security tests (CCTV redirect, GDELT https, sentinel
cache, crowdthreat opt-in, third-party fetcher gates, control
surface auth) continue to pass.
Pre-existing test infrastructure issue with SHARED_EXECUTOR teardown
in the broader sweep exists on main too (verified) — not introduced
by this PR.
Credit: @tg12 reported every one of these with accurate line citations
and the recommended fixes that informed this implementation.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Ship the v0.9.79 runtime refresh with transport lane isolation, Infonet secure-message address management, MeshChat MQTT controls, selected asset trail behavior, telemetry panel refinements, onboarding updates, and desktop/package metadata alignment.
Also ignore local graphify work products so analysis folders do not leak into future commits.
Gate messages now propagate via the Infonet hashchain as encrypted blobs — every node syncs them
through normal chain sync while only Gate members with MLS keys can decrypt. Added mesh reputation
system, peer push workers, voluntary Wormhole opt-in for node participation, fork recovery,
killwormhole scripts, obfuscated terminology, and hardened the self-updater to protect encryption
keys and chain state during updates.
New features: Shodan search, train tracking, Sentinel Hub imagery, 8 new intelligence layers,
CCTV expansion to 11,000+ cameras across 6 countries, Mesh Terminal CLI, prediction markets,
desktop-shell scaffold, and comprehensive mesh test suite (215 frontend + backend tests passing).
Community contributors: @wa1id, @AlborzNazari, @adust09, @Xpirix, @imqdcr, @csysp, @suranyami,
@chr0n1x, @johan-martensson, @singularfailure, @smithbh, @OrfeoTerkuci, @deuza, @tm-const,
@Elhard1, @ttulttul
Auto-update POST goes through Next.js proxy which dies when extracted
files trigger hot-reload. Network drops now transition to restart polling
instead of showing failure. Also adds admin key header and FastAPI error
field fallback. Gitignore updated to protect internal docs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Former-commit-id: 03162f8a4b7ad8a0f2983f81361df7dba42a8689
New features:
- NASA GIBS (MODIS Terra) daily satellite imagery with 30-day time slider
- Esri World Imagery high-res satellite layer (sub-meter, zoom 18+)
- KiwiSDR SDR receivers on map with embedded radio tuner
- Sentinel-2 intel card — right-click for recent satellite photo popup
- LOCATE bar — search by coordinates or place name (Nominatim geocoding)
- SATELLITE style preset in bottom bar cycling
- v0.4 changelog modal on first launch
Fixes:
- Satellite imagery renders below data icons (imagery-ceiling anchor)
- Sentinel-2 opens full-res PNG directly (not STAC catalog JSON)
- Light/dark theme: UI stays dark, only map basemap changes
Security:
- Removed test files with hardcoded API keys from tracking
- Removed .git_backup directory from tracking
- Updated .gitignore to exclude test files, dev scripts, cache files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Former-commit-id: e89e992293
BigBodyCobain