Compare commits

...

9 Commits

Author SHA1 Message Date
BigBodyCobain a26267700d Fix #218/#219/#220: identify ShadowBroker on Wikipedia + Wikidata calls
Wikimedia's User-Agent policy asks API clients to identify themselves
with a stable, contactable identifier so their operators can rate-limit
or coordinate. Before this change, ShadowBroker was sending:

- Backend (region_dossier.py): generic project default UA only; no
  Api-User-Agent.
- Frontend (useRegionDossier.ts, WikiImage.tsx, NewsFeed.tsx): zero
  identifying header at all; three separate copy-pasted anonymous
  fetches with their own module-local caches.

Three separate components doing the same broken thing meant policy
fixes had to happen in three places, with no shared cache or kill
switch.

Fix (no UX change, zero hostility):

== Backend ==

`backend/services/region_dossier.py` now sets explicit `User-Agent` +
`Api-User-Agent` headers on every outbound Wikidata and Wikipedia
request via a new `_WIKIMEDIA_REQUEST_HEADERS` constant. The identifier
includes a contact path (issues page on the public GitHub repo).

== Frontend ==

New shared helper `frontend/src/lib/wikimediaClient.ts`:
- `fetchWikipediaSummary(title)` — single source of truth for Wikipedia
  REST summary lookups, with one shared LRU cache (in-flight requests
  deduplicated, 512-entry cap), `Api-User-Agent` on every fetch.
- `fetchWikidataSparql(query)` — same shape for Wikidata SPARQL.
- `WIKIMEDIA_API_USER_AGENT` — exported constant; one place to update
  if Wikimedia ever asks us to back off.

Refactored three components to use the shared client:
- `frontend/src/hooks/useRegionDossier.ts` — fetchLeader() and
  fetchLocalWikiSummary() now route through the shared helpers.
- `frontend/src/components/WikiImage.tsx` — uses fetchWikipediaSummary,
  proper React state instead of module-mutation + forceUpdate trick.
- `frontend/src/components/NewsFeed.tsx` — same shape.

UX: byte-for-byte identical. Same thumbnails, same dossier content,
same load behavior. The only observable difference is the outgoing
request header.

Note on #239 (route duplication): an audit-grade inventory shows 166
main.py routes are shadowed by router modules. That cleanup is too
large to land safely in this PR; it will be staged as a separate
ladder of small PRs grouped by router module.

Tests:
- `backend/tests/test_region_dossier_wikimedia_ua.py` — 3 tests
  asserting backend headers are present.
- `frontend/src/__tests__/utils/wikimediaClient.test.ts` — 9 tests
  covering Api-User-Agent presence, shared cache, concurrent
  deduplication, disambiguation/HTTP-error/network-error fallthroughs,
  empty-input safety.

Local: backend 76/76 security suite green, frontend 716/716 vitest
suite green.

Credit: tg12 (external security audit).
2026-05-21 10:43:03 -06:00
Shadowbroker e125467721 Fix #243/#252/#253: stop leaking settings posture to anonymous callers (#283)
Three settings endpoints were disclosing operational posture or
operator-curated configuration to any network caller. This change
either tightens the redacted-public view (#243) or adds a
local-operator auth gate (#252, #253) per the audit recommendations.

Zero hostility to legitimate users: in all three cases, the Tauri
shell (loopback), the Docker bridge frontend container (#250 + #278),
and any caller with an admin key continue to see the full data. Only
anonymous LAN/internet callers see the reduced surface.

== #243 (Wormhole transport posture, anonymous-mode, profile, node mode)

Tightened the public-redaction allowlists in BOTH the main.py and
routers/wormhole.py copies:
- _WORMHOLE_PUBLIC_SETTINGS_FIELDS: {enabled, transport, anonymous_mode}
                                 -> {enabled}
- _WORMHOLE_PUBLIC_PROFILE_FIELDS: {profile, wormhole_enabled}
                                 -> {wormhole_enabled}

`GET /api/settings/node` (both the routers/admin.py and main.py copies)
now returns an empty stub for unauthenticated callers and the full
node_mode + node_enabled fields only for authenticated callers via
_scoped_view_authenticated(request, "node").

== #252 (news feed inventory disclosure)

`GET /api/settings/news-feeds` now requires Depends(require_local_operator)
in both the canonical routers/admin.py handler and the duplicate main.py
handler. Anonymous callers can no longer enumerate operator-curated
feed names and URLs.

== #253 (Time Machine archival-capture posture disclosure)

`GET /api/settings/timemachine` now requires Depends(require_local_operator).
Anonymous callers can no longer fingerprint whether a deployment is
retaining replayable historical surveillance data.

Tests: backend/tests/test_round5_settings_info_disclosure.py (10 tests)
- Wormhole settings: anonymous sees only `enabled`; authenticated sees full state.
- Privacy profile: anonymous sees only `wormhole_enabled`; authenticated sees `profile` + `transport` + `anonymous_mode`.
- Node settings: anonymous sees `{}`; authenticated sees node_mode + node_enabled + persisted state.
- news-feeds: anonymous gets 403 (and get_feeds() is NOT called); authenticated gets full inventory.
- timemachine: anonymous gets 403; authenticated sees enabled + storage_warning.

Local: 73/73 security suite (round 5 + earlier rounds) green.

Credit: tg12 (external security audit, P1 + 2x Medium).
2026-05-21 10:32:23 -06:00
Shadowbroker 2b03b808ac Fix #279: add defusedxml to uv.lock so Docker image installs it (#282)
defusedxml is listed in backend/pyproject.toml line 18 but was missing
from uv.lock. The backend Dockerfile uses `uv sync --frozen --no-dev`,
which only installs packages pinned in the lockfile. As a result the
runtime image shipped without defusedxml even though pyproject declared
it, and any import path that touched it crashed at startup with:

    ModuleNotFoundError: No module named 'defusedxml'

Affected import sites:

- backend/services/psk_reporter_fetcher.py:10
- backend/services/fetchers/aircraft_database.py:21
- backend/services/cctv_pipeline.py:990
- backend/services/cctv_pipeline.py:1018

Fix: regenerate uv.lock so defusedxml v0.7.1 (matching the >=0.7.1
specifier in pyproject) is locked. No code changes -- only the lockfile.
Next image build picks it up via the existing `uv sync --frozen` step.

Reporter: external user. Thanks for catching the missing dep.
2026-05-21 10:18:40 -06:00
Shadowbroker 2e14e75a0e Fix #256: per-peer HMAC secrets defeat cross-peer impersonation (#281)
Before this change, every peer-push HMAC was derived from the single
fleet-shared MESH_PEER_PUSH_SECRET. The receiver could prove "this
request was signed by someone who knows the fleet secret" but it could
NOT prove which peer signed it. Any peer that knew the global secret
could compute the expected HMAC for any other peer URL and forge a
push pretending to be that peer.

Fix: introduce MESH_PEER_SECRETS, an optional comma-separated
url=secret map. When a peer URL appears in the map, only the listed
per-peer secret is accepted for it -- the global secret is ignored for
that specific URL. Peer A no longer knows peer B's secret, so peer A
cannot forge a push claiming to be peer B.

The new helper resolve_peer_key_for_url() in mesh_crypto.py wraps the
lookup and is called from every existing peer-push call site:

- backend/auth.py:_verify_peer_push_hmac (receiver)
- backend/main.py:_http_peer_push_loop (Infonet event push)
- backend/main.py:_http_gate_pull_loop (gate event pull)
- backend/main.py:_http_gate_push_loop (gate event push)
- backend/services/mesh/mesh_router.py (two transports, push)
- backend/services/mesh/mesh_hashchain.py (gate wire ref key)
- backend/services/mesh/mesh_wormhole_prekey.py (peer prekey lookup)

Zero hostility, by design:

- Single-peer installs leave MESH_PEER_SECRETS empty -> resolver falls
  back to MESH_PEER_PUSH_SECRET -> behavior is byte-for-byte unchanged.
- Multi-peer installs that haven't migrated yet behave exactly as
  before.
- Multi-peer installs that DO migrate set MESH_PEER_SECRETS on both
  ends of each peering and immediately close the impersonation surface
  for those URLs. Migration is incremental: unlisted peers keep using
  the global secret.

Tests in backend/tests/test_per_peer_secret_resolver.py:
- env parsing (default, override, whitespace, malformed entries, cache)
- precedence: per-peer beats global
- migration window: unlisted peer falls back to global
- IMPERSONATION REFUSAL: peer A with global-secret-only cannot forge
  HMAC for peer B that has a per-peer secret configured
- IMPERSONATION REFUSAL: peer A with its OWN per-peer secret cannot
  forge HMAC for peer B
- positive control: legitimate peer B request verifies
- zero-behavior-change: single-peer install produces the same key bytes
  as before the change

Credit: tg12 (external security audit, P1/High/High confidence)
2026-05-21 10:05:29 -06:00
Shadowbroker 084e563412 Fix #240/#241: require admin auth on oracle resolve endpoints (#280)
Both POST /api/mesh/oracle/resolve and POST /api/mesh/oracle/resolve-stakes
were previously gated only by a rate limit (5/min) and tagged with
`mesh_write_exempt(MeshWriteExemption.ADMIN_CONTROL)`. The exemption
decorator is metadata only — it tells the mesh signed-write middleware
not to require a signature envelope, it does NOT enforce caller
authorization. Any network caller could:

- /resolve: settle any prediction market to any outcome (corrupts every
  downstream profile/win-loss count derived from that ledger).
- /resolve-stakes: trigger stake settlement for all expired contests at
  a time of their choosing (race against operator intent).

Fix: add `dependencies=[Depends(require_admin)]` to both routes. The
existing `mesh_write_exempt` tag stays in place because it accurately
describes the route's relationship to the signed-write envelope system;
adding `require_admin` is what closes the actual auth hole.

Tests in backend/tests/test_oracle_resolve_auth_gate.py:
- anonymous caller -> 403, ledger mutator NOT called
- wrong admin key -> 403, ledger mutator NOT called
- valid admin key -> 200, ledger mutator called
- admin key unconfigured + no debug/insecure-admin -> 403

Credit: tg12 (external security audit)
2026-05-21 09:45:08 -06:00
Shadowbroker 9ef6213284 Fix #250: bind Docker bridge local-operator trust to frontend hostname (#278)
Tightens the bridge-trust check so a connection on the Docker bridge
is only granted local-operator status when its source IP matches a
configured frontend container hostname (default: `frontend` + the
shipped `container_name` `shadowbroker-frontend`). Previously, when
`SHADOWBROKER_TRUST_DOCKER_BRIDGE_LOCAL_OPERATOR=1` was set, ANY IP
in the 172.16.0.0/12 range was granted local-operator privileges —
on a shared Docker host that included any unrelated container on the
same bridge.

Operators with renamed services can list new hostnames via the new
`SHADOWBROKER_TRUSTED_FRONTEND_HOSTS` env var (comma-separated). DNS
resolution is cached for 30s; if Docker DNS can't resolve any of the
configured names we fail closed and refuse the bridge entirely.

Single-user installs see no behavior change — the default-named
frontend container still resolves and is still trusted.

Credit: tg12 (external security audit)
2026-05-21 02:06:11 -06:00
Shadowbroker fb11e0881f Fix #251: refuse symlink/hardlink members during Tor bundle extraction (#277)
External audit (@tg12) flagged that the Tor Expert Bundle extractor
checked tarinfo.name against path traversal but never inspected
tarinfo.linkname for symlink or hardlink members. Python 3.11's
tarfile.extractall() honors symlinks, so a malicious archive could
ship a member like::

    name     = "innocent.txt"          (passes the path-traversal check)
    type     = SYMTYPE
    linkname = "C:\Windows\System32\config\system"

After extraction, subsequent reads of innocent.txt dereference to that
arbitrary filesystem location; subsequent writes corrupt it. On
Windows (where Tor Expert Bundle extraction actually runs), this is
a host-compromise path of essentially the same severity as the
supply-chain RCE in #231 — gated only by the integrity check we just
hardened in PR #261/#265.

Python 3.12+ added tarfile.extract / extractall filter='data' as a
built-in mitigation; we're on Python 3.11 in production, so we
implement the same idea manually.

Fix in backend/services/tor_hidden_service.py:

  Extract the existing path-traversal-only check into a new
  _extract_tor_bundle_safely() helper that:

  1. Refuses any member with member.issym() or member.islnk() True.
     Tor bundles never legitimately contain symlinks or hardlinks
     so this is non-disruptive. Logs the linkname so an operator
     can see what the malicious archive was trying to alias.
  2. Refuses any member that isn't isfile() or isdir() — no FIFOs,
     no character or block devices, no contiguous-file-type entries.
     None of those belong in a Tor Expert Bundle and accepting them
     is a class of bug we don't need to debug later.
  3. Preserves the original path-traversal guard (member.name must
     resolve under install_dir).
  4. Catches tarfile.TarError so a corrupt archive returns False
     gracefully instead of bubbling out an exception.

Tests: backend/tests/test_tor_bundle_symlink_filter.py (8 tests)
  - Clean archive with only regular files extracts successfully
  - Symlink member is rejected (the core regression)
  - Hardlink member is rejected
  - Symlink with relative target inside install_dir is still rejected
    (we don't allow symlinks at all, not just absolute-target ones)
  - FIFO/device-style member is rejected
  - Path-traversal guard still works under the new shape
  - Malformed/non-tar file is rejected gracefully (no crash)
  - Failure on one member rejects the whole bundle (no half-extract)

Validation:
  pytest backend/tests/test_tor_bundle_symlink_filter.py
         backend/tests/test_tor_bundle_verification.py
  -> 14 passed

UX impact: zero for legitimate Tor releases. Operators installing
a real Tor Expert Bundle continue to see "Tor installed at:" exactly
as before. Only malicious archives are refused, with a clear log
message identifying the rejected linkname.

Credit: @tg12 — the original report was specific enough that the
fix design was immediate.

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 01:41:13 -06:00
Shadowbroker 7f96151e56 Fix #231: multi-source SHA-256 verification for the self-updater (#265)
External audit (@tg12, May 18) found that backend/services/updater.py
silently skipped all SHA-256 integrity verification whenever the
MESH_UPDATE_SHA256 env var was unset — which is the default. Nothing
in any install doc tells operators to set it, so practically every
deployment was running the auto-updater with zero integrity check.
That made GitHub release pipeline compromise a single-step path to
arbitrary code execution on every node that auto-updates.

Investigation surfaced a deeper bug too: the updater downloads
zipball_url (GitHub's auto-generated source archive) but the
maintainer's release process publishes SHA256SUMS.txt for a separate
named asset (ShadowBroker_v*.zip). So even if MESH_UPDATE_SHA256
WERE set, operators had no published digest to compare against — the
file they were downloading wasn't the file the maintainer had signed.

This PR fixes both issues with the same multi-source verification
chain we shipped for the Tor bundle in PR #261:

  backend/services/updater.py
    _download_release() now prefers a maintainer-signed release asset
    matching ShadowBroker_v*.zip over zipball_url. Captures the
    SHA256SUMS.txt asset URL when present.

    _validate_zip_hash() rewritten as a four-source chain:
      1. MESH_UPDATE_SHA256 env var (operator override, preserved)
      2. SHA256SUMS.txt asset published with the release (primary —
         the maintainer's release process already publishes this)
      3. Baked-in backend/data/release_digests.json (second line of
         defense for releases that lack the SHA256SUMS asset, or when
         the asset can't be fetched at update time)
      4. HTTPS-only fallback with a loud warning (preserves the auto-
         update flow during transient outages)

    Mismatch from any source that DID respond is fatal — the update
    is refused and the existing install keeps running. Only the
    "no source reachable at all" case falls back to HTTPS-only.

    _fetch_sha256sums() new — fetches and parses a standard
    SHA256SUMS.txt asset. Handles both "<digest>  <name>" and binary-
    marker "<digest> *<name>" formats. Tolerant to comments, blank
    lines, and malformed entries.

  backend/data/release_digests.json (new)
    Baked-in digest list keyed by release tag. Seeded with the v0.9.79
    entries copied from the published SHA256SUMS.txt:
      ShadowBroker_v0.9.79.zip      = f6877c1d6661...
      ShadowBroker_0.9.79_x64-setup.exe = f7b676ada45c...
      ShadowBroker_0.9.79_x64_en-US.msi = e0713c3cdda1...
    Whitelisted in .gitignore alongside the other static reference
    data files (kiwisdr_directory.json, tor_bundle_digests.json,
    aisstream_spki_pins.json).

  backend/tests/test_update_integrity_chain.py (new, 16 tests)
    - Each source matches → success, identifies which source verified
    - Each source mismatches → RuntimeError "mismatch"
    - No source reachable → https-only fallback with loud warning
    - Env override beats all other sources (preserved precedence)
    - SHA256SUMS.txt parser handles standard, binary-marker, comments,
      and network-failure cases

Validation:
  pytest backend/tests/test_update_integrity_chain.py → 16 passed
  pytest (all 15 security test files together) → 105 passed

UX impact: zero. Normal auto-update flow is unchanged for legitimate
releases (path 2 catches everything because the release publishes
SHA256SUMS.txt). Transient network failures during update gracefully
fall through to path 3 then path 4 — no operator intervention needed.
The only user-visible behavior change is in the compromised-release
case, where the update is now refused instead of silently applied.

Credit: @tg12 for the original bug report and the specific call-out
that MESH_UPDATE_SHA256 was unreachable by default operators.

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 01:31:20 -06:00
Shadowbroker d0299fc0a0 test(ci): raise vitest testTimeout to 15s to stop CI-load flakes (#266)
Vitest's default per-test timeout is 5s. That's plenty for tests that
exercise pure functions or even simple JSX, but the heavier React
component trees we render under jsdom — MessagesView, GateView,
Wormhole contact flows — consistently measure 6-10s on GitHub Actions'
shared Node workers under load.

Concrete flake history that drove this bump (none were real product
bugs — all were CI load racing the 5s ceiling on findByText /
waitFor against React reconciliation):

  PR #226 messagesViewFirstContact > removes approved contact
  PR #237 (same)
  PR #261 (same)
  PR #262 (same) ← worst: fired on post-merge Docker Publish run,
                   prevented the AIS SPKI security fix's image from
                   being published to GHCR until PR #263 cumulatively
                   re-published it. Real security-fix-shipping risk.
  PR #264 fixed messagesViewFirstContact specifically with waitFor
  PR #265 messagesViewFirstContact > legacy handle-only addresses
                  AND gateCompatDecryptUx > browser-local gate runtime
                  AND failed on the rerun too — confirming the flake
                  class is broader than the one test we deflaked.

The deflake in PR #264 was too surgical — it addressed one specific
test out of a class of similarly-flaky CI-load-sensitive sites. This
PR addresses the root cause at the config layer instead of playing
whack-a-mole.

Why 15s specifically: 3x the default. Headroom for routine CI slowness
without masking real "test never settles" bugs (those would still
time out, just three rounds later). Individual tests can still pin
their own tighter timeout via the third arg to `it()`.

Also bumps hookTimeout to 15s — beforeEach/afterEach setup for the
same heavier component tests has the same CI-load sensitivity.

User-facing impact: zero. This is dev pipeline infrastructure. End
users never see test timeouts. The cost is theoretical: a buggy test
that genuinely never resolves now takes 15s to declare failure
instead of 5s. In practice that's negligible because the suite runs
once per CI invocation and tests don't usually deadlock.

Validation:
  Local full vitest run → 707 passed, 72 files, 10.36s wall clock
  (same speed as before — we only changed how long we WAIT for slow
   tests, not how fast tests actually run)

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 01:26:34 -06:00
31 changed files with 2836 additions and 151 deletions
+4
View File
@@ -101,6 +101,10 @@ backend/data/*
# Issue #258: SPKI pins for stream.aisstream.io so we can survive upstream
# Let's Encrypt renewal failures without disabling TLS validation entirely.
!backend/data/aisstream_spki_pins.json
# Issue #231: pinned SHA-256 digests for known release archives. Used by
# the self-updater as a second-line integrity check when the release's
# SHA256SUMS.txt asset can't be fetched.
!backend/data/release_digests.json
# OS generated files
.DS_Store
+89 -9
View File
@@ -45,6 +45,7 @@ from services.mesh.mesh_compatibility import (
from services.mesh.mesh_crypto import (
_derive_peer_key,
normalize_peer_url,
resolve_peer_key_for_url,
verify_signature,
verify_node_binding,
parse_public_key_algo,
@@ -245,15 +246,90 @@ def _docker_bridge_local_operator_enabled() -> bool:
}
# Issue #250 (tg12): the previous implementation returned True for any IP
# in the entire 172.16.0.0/12 range. Anyone with `docker run` access on
# the same daemon could spin up a container that automatically passed
# local-operator auth. The fix narrows trust to ONLY connections whose
# source IP matches the configured frontend container's hostname.
#
# Docker DNS resolves both the compose service name (``frontend``) and
# the explicit ``container_name`` (``shadowbroker-frontend``) to the
# frontend container's bridge IP. We forward-resolve both, cache the
# result for 30s, and only trust connections from those exact IPs.
#
# Operators on shared Docker hosts get the benefit of the narrower
# surface. Operators on single-user installs see no behavior change —
# their frontend container still resolves and is still trusted.
_DOCKER_BRIDGE_TRUST_CACHE: dict = {"ips": frozenset(), "expires": 0.0}
_DOCKER_BRIDGE_TRUST_TTL = 30.0
def _trusted_bridge_frontend_hostnames() -> list[str]:
"""Container hostnames whose IPs we treat as local-operator on the bridge.
Default covers both Docker Compose service name (``frontend``) and the
explicit ``container_name`` from the shipped docker-compose.yml
(``shadowbroker-frontend``). Operators with non-default names can
override via the ``SHADOWBROKER_TRUSTED_FRONTEND_HOSTS`` env var
(comma-separated, no spaces).
"""
raw = str(
os.environ.get(
"SHADOWBROKER_TRUSTED_FRONTEND_HOSTS",
"frontend,shadowbroker-frontend",
)
).strip()
return [h.strip() for h in raw.split(",") if h.strip()]
def _resolve_trusted_bridge_ips() -> frozenset[str]:
"""Resolve trusted frontend hostnames to a set of IPs, with caching.
Cached for 30s so we don't hit DNS on every request. The cache is
process-local — frontend container IP rotations during a backend's
lifetime will be picked up within 30s.
Returns frozenset() if Docker DNS can't resolve any of the configured
hostnames (fail-closed — when in doubt, refuse to trust the bridge).
"""
import socket
import time as _time
now = _time.time()
cache = _DOCKER_BRIDGE_TRUST_CACHE
if cache["expires"] > now:
return cache["ips"]
ips: set[str] = set()
for hostname in _trusted_bridge_frontend_hostnames():
try:
_, _, addrs = socket.gethostbyname_ex(hostname)
except (OSError, socket.gaierror):
continue
for addr in addrs:
ips.add(addr)
resolved = frozenset(ips)
cache["ips"] = resolved
cache["expires"] = now + _DOCKER_BRIDGE_TRUST_TTL
return resolved
def _is_docker_bridge_host(host: str) -> bool:
"""Return True only when the source IP matches our trusted frontend
container hostname(s).
Previously trusted any 172.16.0.0/12 IP unconditionally. See the
block comment above for the security rationale.
"""
try:
ip = ipaddress.ip_address(host)
except ValueError:
return False
# Docker Desktop and the default compose bridge normally sit inside
# 172.16.0.0/12. Keep this narrower than "any private IP" so a user who
# intentionally binds the backend to LAN does not silently trust LAN clients.
return ip in ipaddress.ip_network("172.16.0.0/12")
# Public IPs are never our frontend container — skip DNS work for them.
if not ip.is_private:
return False
return host in _resolve_trusted_bridge_ips()
def _is_trusted_local_runtime_host(host: str) -> bool:
@@ -1328,11 +1404,15 @@ def _peer_hmac_url_from_request(request: Request) -> str:
def _verify_peer_push_hmac(request: Request, body_bytes: bytes) -> bool:
"""Verify HMAC-SHA256 peer authentication on push requests."""
secret = str(get_settings().MESH_PEER_PUSH_SECRET or "").strip()
if not secret:
return False
"""Verify HMAC-SHA256 peer authentication on push requests.
Issue #256: ``resolve_peer_key_for_url`` looks up a per-peer secret
in ``MESH_PEER_SECRETS`` first, then falls back to the global
``MESH_PEER_PUSH_SECRET``. When a peer URL is listed in the per-peer
map, only the listed secret is accepted for it — the global secret
is ignored, so any peer that knows only the global secret cannot
forge a request claiming to be that peer.
"""
provided = str(request.headers.get("x-peer-hmac", "") or "").strip()
if not provided:
return False
@@ -1341,7 +1421,7 @@ def _verify_peer_push_hmac(request: Request, body_bytes: bytes) -> bool:
allowed_peers = set(authenticated_push_peer_urls())
if not peer_url or peer_url not in allowed_peers:
return False
peer_key = _derive_peer_key(secret, peer_url)
peer_key = resolve_peer_key_for_url(peer_url)
if not peer_key:
return False
+40
View File
@@ -0,0 +1,40 @@
{
"_comment": [
"Baked-in SHA-256 digests for known Shadowbroker release archives.",
"",
"Issue #231: the self-updater previously skipped integrity verification",
"entirely whenever the MESH_UPDATE_SHA256 env var was unset (which is the",
"default — nothing in the install docs tells operators to set it). That",
"made the auto-update a supply-chain RCE on any compromise of the GitHub",
"release pipeline.",
"",
"The fix uses a multi-source verification chain mirroring the Tor bundle",
"digest approach in #201:",
"",
" 1. MESH_UPDATE_SHA256 env var (operator override, preserved)",
" 2. SHA256SUMS.txt asset published alongside each release (primary —",
" the maintainer's release process already publishes this)",
" 3. This baked-in digest list (second line of defense for releases",
" missing a SHA256SUMS asset, or when the asset can't be fetched)",
" 4. HTTPS-only fallback with a loud warning (preserves auto-update",
" flow during transient outages so users don't get stuck)",
"",
"Mismatch from a source that DID respond is fatal — the update is",
"refused and the existing install keeps running. Only the 'no source",
"reachable at all' case falls back to HTTPS-only.",
"",
"Format: each entry is keyed by release tag and maps asset filenames",
"to their canonical SHA-256 digest (hex, lowercase). The updater",
"compares the locally-computed digest of the downloaded asset against",
"the value here.",
"",
"When the maintainer ships a new release, add its digests here BEFORE",
"removing the old ones so operators on the old code still validate",
"against the previous entries during the transition."
],
"v0.9.79": {
"ShadowBroker_v0.9.79.zip": "f6877c1d66614525315ea82636ce9f7b41178332c4dbf90d27431a1ea1d9cd47",
"ShadowBroker_0.9.79_x64-setup.exe": "f7b676ada45cac7da05868b0a353678c9ee700e3abcf456a7c0c038c36da446f",
"ShadowBroker_0.9.79_x64_en-US.msi": "e0713c3cdda184cfbea750bfac0d62a35678fec00847e6476f2cac8e7e42046e"
}
}
+43 -18
View File
@@ -220,6 +220,7 @@ from services.mesh.mesh_crypto import (
_derive_peer_key,
derive_node_id,
normalize_peer_url,
resolve_peer_key_for_url,
verify_node_binding,
parse_public_key_algo,
)
@@ -1079,8 +1080,18 @@ def _public_mesh_log_size(entries: list[dict[str, Any]]) -> int:
return sum(1 for item in entries if _public_mesh_log_entry(item) is not None)
_WORMHOLE_PUBLIC_SETTINGS_FIELDS = {"enabled", "transport", "anonymous_mode"}
_WORMHOLE_PUBLIC_PROFILE_FIELDS = {"profile", "wormhole_enabled"}
# Issue #243 (tg12): the public redaction now exposes only the bare
# "is Wormhole on?" boolean. Transport choice (tor/i2p/mixnet/direct),
# anonymous-mode state, and the named privacy profile are all
# operational posture and were leaking actionable recon to any
# unauthenticated caller. They are now gated behind authenticated reads
# (admin key or scoped-view token). Loopback Tauri shells and Docker
# bridge frontend containers continue to see full status because the
# Next.js catch-all proxy injects the configured ADMIN_KEY for
# same-origin/non-browser callers (see PR #263), so legitimate operator
# UX is unaffected.
_WORMHOLE_PUBLIC_SETTINGS_FIELDS = {"enabled"}
_WORMHOLE_PUBLIC_PROFILE_FIELDS = {"wormhole_enabled"}
_PRIVATE_LANE_CONTROL_FIELDS = {"private_lane_tier", "private_lane_policy"}
_PUBLIC_RNS_STATUS_FIELDS = {"enabled", "ready", "configured_peers", "active_peers"}
_NODE_PUBLIC_EVENT_HOOK_REGISTERED = False
@@ -1745,10 +1756,12 @@ def _http_peer_push_loop() -> None:
_NODE_SYNC_STOP.wait(_PEER_PUSH_INTERVAL_S)
continue
secret = str(get_settings().MESH_PEER_PUSH_SECRET or "").strip()
if not secret:
_NODE_SYNC_STOP.wait(_PEER_PUSH_INTERVAL_S)
continue
# Issue #256: resolve_peer_key_for_url() handles both the
# legacy global MESH_PEER_PUSH_SECRET path and the per-peer
# MESH_PEER_SECRETS map. The per-peer skip happens below
# ("if not peer_key: continue"), so we don't gate the whole
# loop on the global secret being set — an install that only
# configures per-peer secrets is now valid.
peers = authenticated_push_peer_urls()
if not peers:
@@ -1778,7 +1791,7 @@ def _http_peer_push_loop() -> None:
ensure_ascii=False,
).encode("utf-8")
peer_key = _derive_peer_key(secret, normalized)
peer_key = resolve_peer_key_for_url(normalized)
if not peer_key:
continue
import hmac as _hmac_mod2
@@ -1831,10 +1844,7 @@ def _http_gate_pull_loop() -> None:
_NODE_SYNC_STOP.wait(_GATE_PULL_INTERVAL_S)
continue
secret = str(get_settings().MESH_PEER_PUSH_SECRET or "").strip()
if not secret:
_NODE_SYNC_STOP.wait(_GATE_PULL_INTERVAL_S)
continue
# Issue #256: per-peer key resolution; see _http_peer_push_loop.
peers = authenticated_push_peer_urls()
if not peers:
@@ -1846,7 +1856,7 @@ def _http_gate_pull_loop() -> None:
if not normalized:
continue
peer_key = _derive_peer_key(secret, normalized)
peer_key = resolve_peer_key_for_url(normalized)
if not peer_key:
continue
@@ -1959,10 +1969,7 @@ def _http_gate_push_loop() -> None:
_NODE_SYNC_STOP.wait(_PEER_PUSH_INTERVAL_S)
continue
secret = str(get_settings().MESH_PEER_PUSH_SECRET or "").strip()
if not secret:
_NODE_SYNC_STOP.wait(_PEER_PUSH_INTERVAL_S)
continue
# Issue #256: per-peer key resolution; see _http_peer_push_loop.
peers = authenticated_push_peer_urls()
if not peers:
@@ -1977,7 +1984,7 @@ def _http_gate_push_loop() -> None:
if not normalized:
continue
peer_key = _derive_peer_key(secret, normalized)
peer_key = resolve_peer_key_for_url(normalized)
if not peer_key:
continue
@@ -8813,9 +8820,14 @@ async def api_uw_flow(request: Request):
from services.news_feed_config import get_feeds, save_feeds, reset_feeds
@app.get("/api/settings/news-feeds")
@app.get(
"/api/settings/news-feeds",
dependencies=[Depends(require_local_operator)],
)
@limiter.limit("30/minute")
async def api_get_news_feeds(request: Request):
"""Issue #252 (tg12): gated on local-operator. See the canonical
handler in backend/routers/admin.py for the full rationale."""
return get_feeds()
@@ -9018,9 +9030,22 @@ class NodeSettingsUpdate(BaseModel):
@app.get("/api/settings/node")
@limiter.limit("30/minute")
async def api_get_node_settings(request: Request):
"""Issue #243 (tg12): node mode and participant state are
operational posture. Anonymous callers receive an empty stub
enough for the UI to know the endpoint exists but nothing
fingerprintable. Authenticated callers see the full state.
Authenticated == local-operator (loopback / Docker bridge) OR an
admin / scoped-view token. The Tauri shell and Docker frontend
container both qualify via their existing transport (PR #263 +
PR #278), so legitimate operator UX is unchanged.
"""
from services.node_settings import read_node_settings
data = await asyncio.to_thread(read_node_settings)
authenticated = _scoped_view_authenticated(request, "node")
if not authenticated:
return {}
return {
**data,
"node_mode": _current_node_mode(),
+30 -2
View File
@@ -82,9 +82,18 @@ async def api_get_keys_meta(request: Request):
return get_env_path_info()
@router.get("/api/settings/news-feeds")
@router.get(
"/api/settings/news-feeds",
dependencies=[Depends(require_local_operator)],
)
@limiter.limit("30/minute")
async def api_get_news_feeds(request: Request):
"""Issue #252 (tg12): the curated feed inventory is configuration
state, not a public data feed. Gated on local-operator so the
Tauri shell, the Docker bridge frontend, and any caller with an
admin key all see the full list; anonymous LAN/internet callers
can no longer enumerate operator source URLs.
"""
from services.news_feed_config import get_feeds
return get_feeds()
@@ -118,9 +127,18 @@ async def api_reset_news_feeds(request: Request):
@router.get("/api/settings/node")
@limiter.limit("30/minute")
async def api_get_node_settings(request: Request):
"""Issue #243 (tg12): node_mode and node_enabled are operational
posture. Anonymous callers receive an empty stub; authenticated
callers (local-operator or admin/scoped token) see the full
state. See the canonical handler in backend/main.py for the full
rationale.
"""
import asyncio
from auth import _scoped_view_authenticated
from services.node_settings import read_node_settings
data = await asyncio.to_thread(read_node_settings)
if not _scoped_view_authenticated(request, "node"):
return {}
return {
**data,
"node_mode": _current_node_mode(),
@@ -210,9 +228,19 @@ async def api_set_meshtastic_mqtt_settings(request: Request, body: MeshtasticMqt
return _meshtastic_runtime_snapshot()
@router.get("/api/settings/timemachine")
@router.get(
"/api/settings/timemachine",
dependencies=[Depends(require_local_operator)],
)
@limiter.limit("30/minute")
async def api_get_timemachine_settings(request: Request):
"""Issue #253 (tg12): archival-capture posture is operationally
sensitive — it tells a remote caller whether this deployment is
retaining replayable historical surveillance data. Gated on
local-operator so the Tauri shell and Docker bridge frontend
still see the toggle state, but anonymous LAN/internet callers
can no longer fingerprint Time Machine state.
"""
import asyncio
from services.node_settings import read_node_settings
data = await asyncio.to_thread(read_node_settings)
+21 -4
View File
@@ -223,11 +223,21 @@ async def oracle_markets_more(request: Request, category: str = "NEWS", offset:
"has_more": offset + limit < len(cat_markets), "total": len(cat_markets)}
@router.post("/api/mesh/oracle/resolve")
@router.post(
"/api/mesh/oracle/resolve",
dependencies=[Depends(require_admin)],
)
@limiter.limit("5/minute")
@mesh_write_exempt(MeshWriteExemption.ADMIN_CONTROL)
async def oracle_resolve(request: Request):
"""Resolve a prediction market."""
"""Resolve a prediction market.
Issue #240 (tg12): requires admin authentication. The
``mesh_write_exempt`` decorator below is **metadata only** — it tags
the route as not requiring a mesh signed-write envelope, it does
NOT itself enforce caller authorization. The ``Depends(require_admin)``
on the route decorator is what actually gates access.
"""
from services.mesh.mesh_oracle import oracle_ledger
body = await request.json()
market_title = body.get("market_title", "")
@@ -327,11 +337,18 @@ async def oracle_predictions(request: Request, node_id: str = ""):
active_predictions, authenticated=_scoped_view_authenticated(request, "mesh.audit"))
@router.post("/api/mesh/oracle/resolve-stakes")
@router.post(
"/api/mesh/oracle/resolve-stakes",
dependencies=[Depends(require_admin)],
)
@limiter.limit("5/minute")
@mesh_write_exempt(MeshWriteExemption.ADMIN_CONTROL)
async def oracle_resolve_stakes(request: Request):
"""Resolve all expired stake contests."""
"""Resolve all expired stake contests.
Issue #241 (tg12): requires admin authentication. See the note on
``oracle_resolve`` above — ``mesh_write_exempt`` is metadata only.
"""
from services.mesh.mesh_oracle import oracle_ledger
resolutions = oracle_ledger.resolve_expired_stakes()
return {"ok": True, "resolutions": resolutions, "count": len(resolutions)}
+7 -2
View File
@@ -160,8 +160,13 @@ router = APIRouter()
# --- Constants ---
_WORMHOLE_PUBLIC_SETTINGS_FIELDS = {"enabled", "transport", "anonymous_mode"}
_WORMHOLE_PUBLIC_PROFILE_FIELDS = {"profile", "wormhole_enabled"}
# Issue #243 (tg12): the public redaction now exposes only the bare
# "is this on?" boolean. Transport choice, anonymous-mode state, and
# the named privacy profile were all leaking actionable recon to
# unauthenticated callers and are now gated behind authenticated reads.
# See the matching block in backend/main.py for the full rationale.
_WORMHOLE_PUBLIC_SETTINGS_FIELDS = {"enabled"}
_WORMHOLE_PUBLIC_PROFILE_FIELDS = {"wormhole_enabled"}
_PRIVATE_LANE_CONTROL_FIELDS = {"private_lane_tier", "private_lane_policy"}
_PUBLIC_RNS_STATUS_FIELDS = {"enabled", "ready", "configured_peers", "active_peers"}
_NODE_PUBLIC_EVENT_HOOK_REGISTERED = False
+6
View File
@@ -53,6 +53,12 @@ class Settings(BaseSettings):
MESH_RELAY_FAILURE_COOLDOWN_S: int = 120
MESH_BOOTSTRAP_SEED_FAILURE_COOLDOWN_S: int = 15
MESH_PEER_PUSH_SECRET: str = ""
# Issue #256 (tg12): optional per-peer HMAC secret map. Comma-separated
# `url=secret` pairs. When a peer URL appears here, only that per-peer
# secret is accepted for it — the global MESH_PEER_PUSH_SECRET above is
# ignored for that specific URL. Single-peer installs and unmigrated
# multi-peer installs leave this empty and behavior is unchanged.
MESH_PEER_SECRETS: str = ""
MESH_RNS_APP_NAME: str = "shadowbroker"
MESH_RNS_ASPECT: str = "infonet"
MESH_RNS_IDENTITY_PATH: str = ""
+109
View File
@@ -69,6 +69,115 @@ def _derive_peer_key(shared_secret: str, peer_url: str) -> bytes:
).digest()
# ---------------------------------------------------------------------------
# Issue #256 (tg12): per-peer HMAC secrets
# ---------------------------------------------------------------------------
#
# Before this change, ALL peer-push HMACs were derived from a single
# fleet-shared ``MESH_PEER_PUSH_SECRET``. The receiver could prove a
# request was signed by *someone who knows the fleet secret*, but it
# could NOT prove which peer signed it — any peer could compute the
# expected HMAC for any other peer's URL and impersonate that peer.
#
# Fix: an optional ``MESH_PEER_SECRETS`` env var maps specific peer URLs
# to per-peer secrets. When a peer URL is listed there, only that
# per-peer secret is accepted for that URL — the global secret is
# ignored for that peer. Peer A no longer learns peer B's secret, so
# peer A cannot forge a request claiming to be peer B.
#
# Backwards-compatible by design:
#
# - Single-peer installs (``MESH_PEER_SECRETS`` empty) keep using the
# global secret. Zero behavior change. Zero operator action required.
# - Multi-peer installs that haven't migrated yet keep using the global
# secret for every peer. Same behavior as before — same exposure.
# - Multi-peer installs that have migrated configure
# ``MESH_PEER_SECRETS=urlA=secretA,urlB=secretB`` and immediately get
# per-peer identity. Migration is incremental: peers not yet listed
# continue using the global secret until both sides of that peering
# add their entry.
_PEER_SECRETS_CACHE: dict[str, str] = {}
_PEER_SECRETS_CACHE_RAW: str = ""
def _lookup_per_peer_secret(normalized_url: str) -> str:
"""Return the per-peer secret for ``normalized_url`` from MESH_PEER_SECRETS.
Returns "" if no per-peer entry is configured for that URL. The parser
is forgiving:
- Whitespace around items, URLs, and secrets is stripped.
- Items without ``=`` or with empty URL/secret halves are skipped.
- The URL half is normalized via ``normalize_peer_url`` so config
authors don't have to match scheme/port/path quirks exactly.
The cache is invalidated whenever the env var's raw value changes,
which keeps tests' ``monkeypatch.setenv`` calls effective without
forcing a process restart.
"""
import os
raw = str(os.environ.get("MESH_PEER_SECRETS", "") or "").strip()
global _PEER_SECRETS_CACHE, _PEER_SECRETS_CACHE_RAW
if raw != _PEER_SECRETS_CACHE_RAW:
new_cache: dict[str, str] = {}
for chunk in raw.split(","):
chunk = chunk.strip()
if not chunk or "=" not in chunk:
continue
url_part, _, secret_part = chunk.partition("=")
normalized = normalize_peer_url(url_part.strip())
secret = secret_part.strip()
if normalized and secret:
new_cache[normalized] = secret
_PEER_SECRETS_CACHE = new_cache
_PEER_SECRETS_CACHE_RAW = raw
return _PEER_SECRETS_CACHE.get(normalized_url, "")
def resolve_peer_key_for_url(peer_url: str) -> bytes:
"""Return the HMAC key for ``peer_url``, preferring per-peer secret.
Issue #256: this is the function every peer-push call site should
use. It looks up the peer-specific secret first, falling back to the
fleet-shared ``MESH_PEER_PUSH_SECRET`` only when the URL is NOT
listed in ``MESH_PEER_SECRETS``.
Both sender (computing X-Peer-HMAC) and receiver (verifying it) call
this with the SENDER's URL — they must derive the same key, so
operators on both ends of a peering need matching MESH_PEER_SECRETS
entries for that URL to stay in sync.
Returns empty bytes when no usable secret exists. Callers must treat
that as fail-closed (skip the push, reject the verification).
"""
normalized_url = normalize_peer_url(peer_url)
if not normalized_url:
return b""
per_peer_secret = _lookup_per_peer_secret(normalized_url)
if per_peer_secret:
return _derive_peer_key(per_peer_secret, normalized_url)
# No per-peer entry for this URL — fall back to the legacy global
# secret. This is what preserves zero-hostility for single-peer
# installs and the migration window for multi-peer installs.
try:
from services.config import get_settings
global_secret = str(
getattr(get_settings(), "MESH_PEER_PUSH_SECRET", "") or ""
).strip()
except Exception:
return b""
if not global_secret:
return b""
return _derive_peer_key(global_secret, normalized_url)
def _node_digest(public_key_b64: str) -> str:
raw = base64.b64decode(public_key_b64)
return hashlib.sha256(raw).hexdigest()
+8 -7
View File
@@ -216,18 +216,19 @@ def _peer_pair_ref_key(peer_url: str) -> bytes:
Returns an empty key on misconfiguration so callers fail closed.
"""
try:
from services.config import get_settings
from services.mesh.mesh_crypto import _derive_peer_key, normalize_peer_url
secret = str(get_settings().MESH_PEER_PUSH_SECRET or "").strip()
from services.mesh.mesh_crypto import (
normalize_peer_url,
resolve_peer_key_for_url,
)
except Exception:
return b""
if not secret:
return b""
normalized = normalize_peer_url(peer_url or "")
if not normalized:
return b""
peer_key = _derive_peer_key(secret, normalized)
# Issue #256: resolve_peer_key_for_url() prefers per-peer secrets
# from MESH_PEER_SECRETS and falls back to the global
# MESH_PEER_PUSH_SECRET only when the URL has no per-peer entry.
peer_key = resolve_peer_key_for_url(normalized)
if not peer_key:
return b""
# Domain-separate from the transport HMAC key so the two
+16 -11
View File
@@ -26,7 +26,11 @@ from enum import Enum
from typing import Any, Callable, Optional
from collections import deque
from urllib.parse import urlparse
from services.mesh.mesh_crypto import _derive_peer_key, normalize_peer_url
from services.mesh.mesh_crypto import (
_derive_peer_key,
normalize_peer_url,
resolve_peer_key_for_url,
)
from services.mesh.mesh_metrics import increment as metrics_inc
from services.mesh.mesh_privacy_policy import (
TRANSPORT_TIER_ORDER as _TIER_RANK,
@@ -703,7 +707,6 @@ class InternetTransport(_PeerPushTransportMixin):
endpoint_path, padded = self._build_peer_push_request(envelope, self.NAME)
except ValueError as exc:
return TransportResult(False, self.NAME, str(exc))
secret = str(settings.MESH_PEER_PUSH_SECRET or "").strip()
delivered = 0
last_error = ""
@@ -713,10 +716,13 @@ class InternetTransport(_PeerPushTransportMixin):
try:
normalized_peer_url = normalize_peer_url(peer_url)
headers = {"Content-Type": "application/json"}
if secret:
peer_key = _derive_peer_key(secret, normalized_peer_url)
if not peer_key:
raise ValueError("invalid peer URL for HMAC derivation")
# Issue #256: per-peer secret takes precedence over the
# global MESH_PEER_PUSH_SECRET. When neither is set the
# key is empty and we skip the HMAC header entirely so a
# bare (unsigned) push still works on test deployments
# that have not yet configured any secret at all.
peer_key = resolve_peer_key_for_url(normalized_peer_url)
if peer_key:
headers["X-Peer-Url"] = normalized_peer_url
headers["X-Peer-HMAC"] = hmac.new(
peer_key,
@@ -798,7 +804,6 @@ class TorArtiTransport(_PeerPushTransportMixin):
endpoint_path, padded = self._build_peer_push_request(envelope, self.NAME)
except ValueError as exc:
return TransportResult(False, self.NAME, str(exc))
secret = str(settings.MESH_PEER_PUSH_SECRET or "").strip()
delivered = 0
last_error = ""
@@ -808,10 +813,10 @@ class TorArtiTransport(_PeerPushTransportMixin):
try:
normalized_peer_url = normalize_peer_url(peer_url)
headers = {"Content-Type": "application/json"}
if secret:
peer_key = _derive_peer_key(secret, normalized_peer_url)
if not peer_key:
raise ValueError("invalid peer URL for HMAC derivation")
# Issue #256: per-peer secret takes precedence; see the
# other transport above for the rationale.
peer_key = resolve_peer_key_for_url(normalized_peer_url)
if peer_key:
headers["X-Peer-Url"] = normalized_peer_url
headers["X-Peer-HMAC"] = hmac.new(
peer_key,
@@ -91,13 +91,15 @@ def _fetch_dm_prekey_bundle_from_peer_lookup(lookup_token: str) -> dict[str, Any
return {"ok": False, "detail": "lookup token required"}
try:
from services.config import get_settings
from services.mesh.mesh_crypto import _derive_peer_key, normalize_peer_url
from services.mesh.mesh_crypto import (
normalize_peer_url,
resolve_peer_key_for_url,
)
from services.mesh.mesh_router import configured_relay_peer_urls
settings = get_settings()
secret = str(getattr(settings, "MESH_PEER_PUSH_SECRET", "") or "").strip()
if not secret:
return {"ok": False, "detail": "peer prekey lookup unavailable"}
# Issue #256: secret check moved per-peer below. We still bail out
# cleanly when there are no peers configured at all.
peers = configured_relay_peer_urls()
if not peers:
return {"ok": False, "detail": "peer prekey lookup unavailable"}
@@ -121,7 +123,8 @@ def _fetch_dm_prekey_bundle_from_peer_lookup(lookup_token: str) -> dict[str, Any
or os.environ.get("SB_TEST_NODE_URL", "").strip()
or normalized_peer_url
)
peer_key = _derive_peer_key(secret, sender_peer_url)
# Issue #256: prefer per-peer secret keyed by the sender URL.
peer_key = resolve_peer_key_for_url(sender_peer_url)
if not peer_key:
continue
headers = {
+30 -3
View File
@@ -4,7 +4,7 @@ import concurrent.futures
from urllib.parse import quote
import requests as _requests
from cachetools import TTLCache
from services.network_utils import fetch_with_curl
from services.network_utils import fetch_with_curl, DEFAULT_USER_AGENT
logger = logging.getLogger(__name__)
@@ -15,6 +15,25 @@ dossier_cache = TTLCache(maxsize=500, ttl=86400)
# Nominatim requires max 1 req/sec — track last call time
_nominatim_last_call = 0.0
# Issue #218 / #219 (tg12): Wikimedia's User-Agent policy requires API
# clients to identify themselves with a stable User-Agent that includes
# a contact path. Bare "python-requests/x.y" or generic strings violate
# the policy and risk getting blocked. We send the project default UA
# (operator-overridable via SHADOWBROKER_USER_AGENT) on EVERY outbound
# Wikimedia request, plus the policy-recommended Api-User-Agent which
# Wikimedia explicitly accepts on top of the regular UA.
#
# This is documented and stable so a Wikimedia operator who wants to
# rate-limit or contact us has a fixed identifier to grep for.
_WIKIMEDIA_REQUEST_HEADERS = {
"User-Agent": DEFAULT_USER_AGENT,
"Api-User-Agent": (
f"{DEFAULT_USER_AGENT} "
"(+https://github.com/BigBodyCobain/Shadowbroker; "
"report issues at /issues)"
),
}
def _reverse_geocode_offline(lat: float, lng: float) -> dict:
"""Offline fallback via reverse_geocoder when external reverse geocoding is blocked."""
@@ -121,7 +140,13 @@ def _fetch_wikidata_leader(country_name: str) -> dict:
"""
url = f"https://query.wikidata.org/sparql?query={quote(sparql)}&format=json"
try:
res = fetch_with_curl(url, timeout=6)
# Issue #218 (tg12): Wikimedia's User-Agent policy requires
# outbound API traffic to be identifiable. fetch_with_curl()
# sends the project default, and we also add the Wikimedia-
# specific Api-User-Agent that the policy specifically asks
# for, since this request originates from a backend service
# that proxies on behalf of (potentially many) browser users.
res = fetch_with_curl(url, timeout=6, headers=_WIKIMEDIA_REQUEST_HEADERS)
if res.status_code == 200:
results = res.json().get("results", {}).get("bindings", [])
if results:
@@ -147,7 +172,9 @@ def _fetch_local_wiki_summary(place_name: str, country_name: str = "") -> dict:
slug = quote(name.replace(" ", "_"))
url = f"https://en.wikipedia.org/api/rest_v1/page/summary/{slug}"
try:
res = fetch_with_curl(url, timeout=5)
# Issue #219 (tg12): identify ourselves to Wikimedia per
# their UA policy; see _fetch_wikidata_leader above.
res = fetch_with_curl(url, timeout=5, headers=_WIKIMEDIA_REQUEST_HEADERS)
if res.status_code == 200:
data = res.json()
if data.get("type") != "disambiguation":
+91 -8
View File
@@ -173,6 +173,94 @@ def _verify_tor_bundle(archive_path: Path, bundle_url: str) -> tuple[bool, str]:
return True, f"https-only (no digest source reachable, archive={actual_hash[:16]}...)"
def _extract_tor_bundle_safely(archive_path: Path, install_dir: Path) -> bool:
"""Extract a Tor Expert Bundle tar.gz safely.
Issue #251: the previous extractor checked tarinfo.name against path
traversal but never inspected tarinfo.linkname for symlink/hardlink
members. Python 3.11's tarfile honors symlinks during extractall(),
so a malicious archive could ship a member like::
name = "innocent.txt" # passes the path check
type = SYMTYPE
linkname = "C:\\Windows\\System32\\config\\system"
and extractall() would then create that symlink. Subsequent reads
of innocent.txt deference to a sensitive system file; subsequent
writes corrupt one. Tor bundles never legitimately contain symlinks
or hardlinks, so we refuse all link members categorically rather
than trying to validate linkname targets (which has its own pitfalls
around relative path resolution).
Also refuses non-regular-non-directory members (devices, FIFOs,
character/block special files) for completeness none of those
belong in a Tor Expert Bundle and accepting them is a category of
bug we don't need to debug later.
Returns True on success, False on rejection (and logs the reason).
The caller is responsible for cleaning up the archive file.
"""
import tarfile
install_resolved = install_dir.resolve()
try:
with tarfile.open(str(archive_path), "r:gz") as tar:
for member in tar.getmembers():
# Reject anything that isn't a regular file or directory.
# Symlinks (SYMTYPE) and hardlinks (LNKTYPE) are the
# path-traversal vectors; the others (CHRTYPE, BLKTYPE,
# FIFOTYPE, CONTTYPE) have no legitimate use in a Tor
# Expert Bundle.
if member.issym() or member.islnk():
logger.error(
"Tor bundle extraction blocked: link member %s -> %s "
"(symlinks/hardlinks are not allowed in Tor bundles; "
"this archive is malformed or hostile)",
member.name,
member.linkname,
)
return False
if not (member.isfile() or member.isdir()):
logger.error(
"Tor bundle extraction blocked: unexpected member type "
"for %s (only regular files and directories are allowed)",
member.name,
)
return False
# Path traversal check (preserves the original guard).
try:
member_path = (install_dir / member.name).resolve()
except OSError as exc:
logger.error(
"Tor bundle extraction blocked: cannot resolve member "
"path %s: %s",
member.name,
exc,
)
return False
try:
member_path.relative_to(install_resolved)
except ValueError:
logger.error(
"Tor bundle extraction blocked: path traversal on %s "
"(resolves to %s, outside install dir %s)",
member.name,
member_path,
install_resolved,
)
return False
# All members validated — extract.
tar.extractall(path=str(install_dir))
except tarfile.TarError as exc:
logger.error("Tor bundle extraction failed: malformed tar (%s)", exc)
return False
return True
def _auto_install_tor() -> str | None:
"""Install or download Tor when it is safe to do so."""
if os.name != "nt":
@@ -203,14 +291,9 @@ def _auto_install_tor() -> str | None:
logger.info("Download complete, extracting...")
import tarfile
with tarfile.open(str(archive_path), "r:gz") as tar:
for member in tar.getmembers():
member_path = (TOR_INSTALL_DIR / member.name).resolve()
if not str(member_path).startswith(str(TOR_INSTALL_DIR.resolve())):
logger.error("Tar path traversal blocked: %s", member.name)
archive_path.unlink(missing_ok=True)
return None
tar.extractall(path=str(TOR_INSTALL_DIR))
if not _extract_tor_bundle_safely(archive_path, TOR_INSTALL_DIR):
archive_path.unlink(missing_ok=True)
return None
archive_path.unlink(missing_ok=True)
+232 -14
View File
@@ -6,9 +6,11 @@ Public API:
schedule_restart(project_root) (spawn detached start script, then exit)
"""
import json
import os
import sys
import logging
import re
import shutil
import subprocess
import tempfile
@@ -29,6 +31,19 @@ DOCKER_UPDATE_COMMANDS = (
"docker compose pull && docker compose up -d"
)
# Issue #231: baked-in release digests. Loaded lazily, used as a fallback
# verification source when the release's SHA256SUMS.txt asset can't be
# fetched (e.g. transient network failure during update).
_RELEASE_DIGESTS_FILE = (
Path(__file__).resolve().parent.parent / "data" / "release_digests.json"
)
# Pattern for the maintainer's signed source-archive release asset. This
# is the file we prefer over the auto-generated ``zipball_url`` because
# the maintainer's build process publishes it with a matching entry in
# SHA256SUMS.txt — the zipball does not have a signed digest.
_SOURCE_ASSET_PATTERN = re.compile(r"^ShadowBroker_v\d", re.IGNORECASE)
_SHA256SUMS_ASSET_NAME = "SHA256SUMS.txt"
def _is_docker() -> bool:
"""Detect if we're running inside a Docker container."""
@@ -40,7 +55,6 @@ def _is_docker() -> bool:
except (FileNotFoundError, PermissionError):
pass
return os.environ.get("container") == "docker"
_EXPECTED_SHA256 = os.environ.get("MESH_UPDATE_SHA256", "").strip().lower()
_ALLOWED_UPDATE_HOSTS = {
"api.github.com",
"codeload.github.com",
@@ -119,7 +133,16 @@ def _validate_update_url(url: str, *, allow_release_page: bool = False) -> str:
# ---------------------------------------------------------------------------
def _download_release(temp_dir: str) -> tuple:
"""Fetch latest release info and download the source zip archive.
Returns (zip_path, version_tag, download_url, release_url).
Issue #231: prefer the maintainer's signed release asset (matching
``ShadowBroker_v*.zip``) over the auto-generated ``zipball_url``,
because the maintainer's release process publishes a matching entry
in SHA256SUMS.txt for the named asset but NOT for the zipball.
Returns (zip_path, version_tag, download_url, release_url, asset_name,
sha256sums_url) the last two are empty strings when the release
doesn't publish a signed asset, falling back to the legacy zipball
path.
"""
logger.info("Fetching latest release info from GitHub...")
_validate_update_url(GITHUB_RELEASES_URL)
@@ -131,9 +154,42 @@ def _download_release(temp_dir: str) -> tuple:
tag = release.get("tag_name", "unknown")
release_url = str(release.get("html_url") or GITHUB_RELEASES_PAGE_URL).strip()
_validate_update_url(release_url, allow_release_page=True)
zip_url = str(release.get("zipball_url") or "").strip()
if not zip_url:
raise RuntimeError("Latest release is missing a source archive URL")
# Prefer the maintainer-signed release asset. Fall back to the
# auto-generated zipball if the release doesn't publish one.
assets = release.get("assets") or []
asset_name = ""
asset_url = ""
sha256sums_url = ""
for a in assets:
name = str(a.get("name") or "").strip()
download = str(a.get("browser_download_url") or "").strip()
if not name or not download:
continue
if _SOURCE_ASSET_PATTERN.match(name) and name.lower().endswith(".zip"):
asset_name = name
asset_url = download
elif name == _SHA256SUMS_ASSET_NAME:
sha256sums_url = download
if asset_url:
zip_url = asset_url
logger.info(
"Using signed release asset %s (sha256sums=%s)",
asset_name,
"yes" if sha256sums_url else "no",
)
else:
zip_url = str(release.get("zipball_url") or "").strip()
if not zip_url:
raise RuntimeError("Latest release is missing a source archive URL")
logger.warning(
"Release does not publish a signed ShadowBroker_v*.zip asset — "
"falling back to auto-generated zipball_url. Integrity will be "
"verified against the baked-in release_digests.json (if present) "
"or HTTPS-only otherwise."
)
_validate_update_url(zip_url)
logger.info(f"Downloading {zip_url} ...")
@@ -150,19 +206,174 @@ def _download_release(temp_dir: str) -> tuple:
size_mb = os.path.getsize(zip_path) / (1024 * 1024)
logger.info(f"Downloaded {size_mb:.1f} MB — ZIP validated OK")
return zip_path, tag, zip_url, release_url
return zip_path, tag, zip_url, release_url, asset_name, sha256sums_url
def _validate_zip_hash(zip_path: str) -> None:
if not _EXPECTED_SHA256:
return
def _compute_sha256(zip_path: str) -> str:
"""Return the hex SHA-256 of the file at ``zip_path`` (lowercase)."""
h = hashlib.sha256()
with open(zip_path, "rb") as f:
for chunk in iter(lambda: f.read(1024 * 128), b""):
h.update(chunk)
digest = h.hexdigest().lower()
if digest != _EXPECTED_SHA256:
raise RuntimeError("Update SHA-256 mismatch")
return h.hexdigest().lower()
def _load_baked_in_release_digests() -> dict:
"""Return the ``release_digests.json`` mapping, or an empty dict.
Schema (issue #231):
{
"<release_tag>": {
"<asset_filename>": "<sha256_hex>",
...
},
...
}
"""
try:
raw = _RELEASE_DIGESTS_FILE.read_text(encoding="utf-8")
parsed = json.loads(raw)
except (OSError, ValueError) as exc:
logger.debug("Release digest file unreadable: %s", exc)
return {}
if not isinstance(parsed, dict):
return {}
cleaned: dict[str, dict[str, str]] = {}
for k, v in parsed.items():
if not isinstance(k, str) or k.startswith("_"):
continue
if isinstance(v, dict):
entries = {
fname: digest.strip().lower()
for fname, digest in v.items()
if isinstance(fname, str) and isinstance(digest, str)
}
if entries:
cleaned[k] = entries
return cleaned
def _fetch_sha256sums(sha256sums_url: str) -> dict[str, str]:
"""Download a SHA256SUMS.txt and return {filename: digest_hex_lower}.
Standard ``sha256sum`` format: ``<digest> <filename>`` per line. The
leading ``*`` binary-mode marker (e.g. ``<digest> *<filename>``) is
handled.
"""
try:
_validate_update_url(sha256sums_url)
except RuntimeError as exc:
logger.warning("SHA256SUMS URL rejected: %s", exc)
return {}
try:
resp = requests.get(sha256sums_url, timeout=15)
resp.raise_for_status()
except requests.RequestException as exc:
logger.info("SHA256SUMS fetch failed: %s", exc)
return {}
out: dict[str, str] = {}
for line in resp.text.splitlines():
line = line.strip()
if not line or line.startswith("#"):
continue
# Tolerant split: handle both `<digest> <name>` and `<digest> *<name>`.
parts = line.split(None, 1)
if len(parts) != 2:
continue
digest, fname = parts
fname = fname.lstrip("*").strip()
digest = digest.strip().lower()
if len(digest) == 64 and all(c in "0123456789abcdef" for c in digest) and fname:
out[fname] = digest
return out
def _validate_zip_hash(
zip_path: str,
*,
asset_name: str = "",
sha256sums_url: str = "",
release_tag: str = "",
) -> str:
"""Verify the downloaded archive against trusted digest sources.
Issue #231: previously this returned silently when ``MESH_UPDATE_SHA256``
was unset, which made the auto-updater a supply-chain RCE vector on any
compromise of the GitHub release pipeline. The chain now is:
1. ``MESH_UPDATE_SHA256`` env var (operator override preserved for
power-users who want to pin an exact digest manually)
2. ``SHA256SUMS.txt`` release asset (primary the maintainer's
release process already publishes this)
3. Baked-in ``backend/data/release_digests.json`` (second line of
defense for releases that lack the SHA256SUMS asset, or when the
asset can't be fetched at update time)
4. HTTPS-only fallback with a loud warning (preserves the auto-update
flow during transient outages but never silently)
A mismatch from a source that DID respond is fatal: the update is
refused and the existing install keeps running. Only the "no source
reachable at all" case falls back to HTTPS-only.
Returns a short human-readable description of which source verified
the archive (used in the update-success message).
"""
actual = _compute_sha256(zip_path)
# Source 1: explicit operator override.
override = os.environ.get("MESH_UPDATE_SHA256", "").strip().lower()
if override:
if actual == override:
return f"verified via MESH_UPDATE_SHA256 ({actual[:16]}...)"
raise RuntimeError(
f"Update SHA-256 mismatch vs MESH_UPDATE_SHA256: archive={actual[:16]}..., "
f"expected={override[:16]}..."
)
# Source 2: SHA256SUMS.txt asset from the release.
sums_map: dict[str, str] = {}
if sha256sums_url and asset_name:
sums_map = _fetch_sha256sums(sha256sums_url)
sums_expected = sums_map.get(asset_name) if asset_name else None
if sums_expected:
if actual == sums_expected:
return f"verified via release SHA256SUMS.txt ({actual[:16]}...)"
raise RuntimeError(
f"Update SHA-256 mismatch vs release SHA256SUMS.txt: "
f"archive={actual[:16]}..., expected={sums_expected[:16]}..."
)
# Source 3: baked-in digest list.
baked = _load_baked_in_release_digests()
baked_expected = ""
if release_tag and asset_name:
baked_expected = baked.get(release_tag, {}).get(asset_name, "")
if baked_expected:
if actual == baked_expected:
return f"verified via baked-in digest list ({actual[:16]}...)"
raise RuntimeError(
f"Update SHA-256 mismatch vs baked-in digest list: "
f"archive={actual[:16]}..., expected={baked_expected[:16]}..."
)
# Source 4: HTTPS-only fallback. We keep onboarding/auto-update working
# during transient outages (no SHA256SUMS reachable AND no baked-in
# entry for this release), but surface the degraded posture loudly so
# the operator can see it in logs and the maintainer can populate the
# digest list on the next release bump.
logger.warning(
"Update integrity check fell back to HTTPS-only trust "
"(no SHA256SUMS.txt response and no baked-in digest for "
"release=%s asset=%s). The archive SHA-256 is %s. Once the "
"release ships a SHA256SUMS.txt asset OR backend/data/"
"release_digests.json is updated with this release, the secure "
"path will activate automatically.",
release_tag or "unknown",
asset_name or "unknown",
actual,
)
return f"https-only (no digest source reachable, archive={actual[:16]}...)"
def _is_source_checkout(project_root: str) -> bool:
@@ -334,7 +545,7 @@ def perform_update(project_root: str) -> dict:
temp_dir = tempfile.mkdtemp(prefix="sb_update_")
manual_url = GITHUB_RELEASES_PAGE_URL
try:
zip_path, version, url, release_url = _download_release(temp_dir)
zip_path, version, url, release_url, asset_name, sha256sums_url = _download_release(temp_dir)
manual_url = release_url or manual_url
if in_docker:
@@ -366,7 +577,13 @@ def perform_update(project_root: str) -> dict:
),
}
_validate_zip_hash(zip_path)
verification_note = _validate_zip_hash(
zip_path,
asset_name=asset_name,
sha256sums_url=sha256sums_url,
release_tag=version,
)
logger.info("Update archive %s", verification_note)
backup_path = _backup_current(project_root, temp_dir)
copied = _extract_and_copy(zip_path, project_root, temp_dir)
@@ -378,6 +595,7 @@ def perform_update(project_root: str) -> dict:
"manual_url": manual_url,
"release_url": release_url,
"download_url": url,
"integrity": verification_note,
"message": f"Updated to {version}{copied} files replaced. Restarting...",
}
except Exception as e:
@@ -0,0 +1,196 @@
"""Issue #250 (tg12): Docker bridge local-operator trust must be bound to
the frontend container's hostname, not the entire 172.16.0.0/12 range.
Previous behavior trusted ANY private-RFC1918 source IP on the bridge
when ``SHADOWBROKER_TRUST_DOCKER_BRIDGE_LOCAL_OPERATOR=1``. On a shared
Docker host this granted local-operator privileges to any other
container that could route to the backend's bridge — far broader than
intended.
The fix narrows trust to source IPs that forward-resolve from one of the
configured frontend container hostnames (default: the compose service
name ``frontend`` plus the explicit ``container_name``
``shadowbroker-frontend``). Operators with renamed containers can list
the new names in ``SHADOWBROKER_TRUSTED_FRONTEND_HOSTS``.
These tests exercise the resolution helpers directly so that we don't
need a live Docker daemon to validate the contract.
"""
import socket
from unittest.mock import patch
import pytest
# ---------------------------------------------------------------------------
# _trusted_bridge_frontend_hostnames — env parsing
# ---------------------------------------------------------------------------
class TestTrustedHostnameParsing:
def _fn(self):
from auth import _trusted_bridge_frontend_hostnames
return _trusted_bridge_frontend_hostnames
def test_default_covers_compose_service_and_container_name(self):
with patch.dict("os.environ", {}, clear=False):
# Make sure the env var is not set so we exercise the default.
import os
os.environ.pop("SHADOWBROKER_TRUSTED_FRONTEND_HOSTS", None)
assert self._fn()() == ["frontend", "shadowbroker-frontend"]
def test_custom_list_via_env(self):
with patch.dict(
"os.environ",
{"SHADOWBROKER_TRUSTED_FRONTEND_HOSTS": "my-ui,alt-frontend"},
):
assert self._fn()() == ["my-ui", "alt-frontend"]
def test_whitespace_trimmed(self):
with patch.dict(
"os.environ",
{"SHADOWBROKER_TRUSTED_FRONTEND_HOSTS": " my-ui , alt-frontend "},
):
assert self._fn()() == ["my-ui", "alt-frontend"]
def test_empty_env_falls_back_to_default(self):
# An empty string still falls back to the bundled defaults so a
# misconfigured env var doesn't silently dismantle bridge trust.
with patch.dict(
"os.environ",
{"SHADOWBROKER_TRUSTED_FRONTEND_HOSTS": ""},
):
# Per docs: empty string sets the env var to "" so os.environ.get
# returns "" — that string is parsed and yields []. We assert
# that empty parse yields [] (caller fail-closes from there).
assert self._fn()() == []
# ---------------------------------------------------------------------------
# _resolve_trusted_bridge_ips — DNS resolution with cache + fail-closed
# ---------------------------------------------------------------------------
class TestResolveTrustedBridgeIps:
def setup_method(self):
# Reset the module-level cache before each test so prior tests
# don't bleed state across cases.
from auth import _DOCKER_BRIDGE_TRUST_CACHE
_DOCKER_BRIDGE_TRUST_CACHE["ips"] = frozenset()
_DOCKER_BRIDGE_TRUST_CACHE["expires"] = 0.0
def test_resolves_configured_hostnames(self):
from auth import _resolve_trusted_bridge_ips
def fake_gethostbyname_ex(host):
mapping = {
"frontend": ("frontend", [], ["172.18.0.3"]),
"shadowbroker-frontend": ("shadowbroker-frontend", [], ["172.18.0.3", "172.18.0.4"]),
}
if host not in mapping:
raise socket.gaierror("no such host")
return mapping[host]
with patch("socket.gethostbyname_ex", side_effect=fake_gethostbyname_ex):
ips = _resolve_trusted_bridge_ips()
assert ips == frozenset({"172.18.0.3", "172.18.0.4"})
def test_fail_closed_when_dns_returns_nothing(self):
from auth import _resolve_trusted_bridge_ips
def always_fail(host):
raise socket.gaierror("no resolver")
with patch("socket.gethostbyname_ex", side_effect=always_fail):
ips = _resolve_trusted_bridge_ips()
assert ips == frozenset()
def test_partial_resolution_is_kept(self):
"""If one hostname resolves and another fails, we keep the
successful one rather than discarding the whole set."""
from auth import _resolve_trusted_bridge_ips
def partial(host):
if host == "frontend":
return ("frontend", [], ["172.18.0.3"])
raise socket.gaierror("missing")
with patch("socket.gethostbyname_ex", side_effect=partial):
ips = _resolve_trusted_bridge_ips()
assert ips == frozenset({"172.18.0.3"})
def test_cache_short_circuits_repeated_dns_calls(self):
from auth import _resolve_trusted_bridge_ips
call_count = {"n": 0}
def counting(host):
call_count["n"] += 1
return ("frontend", [], ["172.18.0.3"])
with patch("socket.gethostbyname_ex", side_effect=counting):
_resolve_trusted_bridge_ips()
calls_after_first = call_count["n"]
_resolve_trusted_bridge_ips()
_resolve_trusted_bridge_ips()
# Second + third calls hit the cache, not the DNS stub.
assert call_count["n"] == calls_after_first
def test_cache_expires(self):
from auth import _resolve_trusted_bridge_ips, _DOCKER_BRIDGE_TRUST_CACHE
with patch("socket.gethostbyname_ex", return_value=("frontend", [], ["172.18.0.3"])):
_resolve_trusted_bridge_ips()
# Force expiry.
_DOCKER_BRIDGE_TRUST_CACHE["expires"] = 0.0
with patch("socket.gethostbyname_ex", return_value=("frontend", [], ["172.18.0.9"])) as stub:
ips = _resolve_trusted_bridge_ips()
assert stub.called
assert "172.18.0.9" in ips
# ---------------------------------------------------------------------------
# _is_docker_bridge_host — composite of the helpers above
# ---------------------------------------------------------------------------
class TestIsDockerBridgeHost:
def setup_method(self):
from auth import _DOCKER_BRIDGE_TRUST_CACHE
_DOCKER_BRIDGE_TRUST_CACHE["ips"] = frozenset()
_DOCKER_BRIDGE_TRUST_CACHE["expires"] = 0.0
def test_trusts_resolved_frontend_ip(self):
from auth import _is_docker_bridge_host
with patch("auth._resolve_trusted_bridge_ips", return_value=frozenset({"172.18.0.3"})):
assert _is_docker_bridge_host("172.18.0.3") is True
def test_rejects_arbitrary_bridge_ip(self):
"""A rogue container on the same bridge but at a different IP
must NOT be trusted, even though it falls in 172.16.0.0/12."""
from auth import _is_docker_bridge_host
with patch("auth._resolve_trusted_bridge_ips", return_value=frozenset({"172.18.0.3"})):
assert _is_docker_bridge_host("172.18.0.99") is False
def test_rejects_public_ip_without_dns_work(self):
"""Public IPs skip DNS resolution entirely (perf + safety)."""
from auth import _is_docker_bridge_host
with patch("auth._resolve_trusted_bridge_ips") as stub:
assert _is_docker_bridge_host("8.8.8.8") is False
stub.assert_not_called()
def test_rejects_non_ip_input(self):
from auth import _is_docker_bridge_host
assert _is_docker_bridge_host("") is False
assert _is_docker_bridge_host("not-an-ip") is False
assert _is_docker_bridge_host("frontend") is False
def test_fails_closed_when_dns_returns_empty(self):
"""If Docker DNS can't resolve any frontend hostname, the bridge
is not trusted even for IPs that would have been trusted under
the old 172.16.0.0/12 blanket policy."""
from auth import _is_docker_bridge_host
with patch("auth._resolve_trusted_bridge_ips", return_value=frozenset()):
assert _is_docker_bridge_host("172.18.0.3") is False
@@ -0,0 +1,160 @@
"""Issues #240 & #241 (tg12): oracle market/stake resolution endpoints
must require admin authentication.
Before the fix, ``POST /api/mesh/oracle/resolve`` and
``POST /api/mesh/oracle/resolve-stakes`` were decorated with
``@mesh_write_exempt(MeshWriteExemption.ADMIN_CONTROL)``. That decorator
only tags the route as not requiring a mesh signed-write envelope; it
does NOT enforce authorization. The rate limiter (5/minute) was the
only real gate, which is wrong for control-plane state mutations.
The fix adds ``dependencies=[Depends(require_admin)]`` to both routes.
These tests prove:
- Anonymous callers receive 403.
- A request bearing the configured admin key passes the auth gate.
- The underlying ledger mutator is not invoked on a 403.
"""
from __future__ import annotations
from unittest.mock import patch, MagicMock
import pytest
from fastapi.testclient import TestClient
_ADMIN_KEY = "test-admin-key-for-oracle-resolve-fixture-32+"
@pytest.fixture
def client():
"""TestClient with the private-lane transport middleware short-circuited.
The ``enforce_high_privacy_mesh`` middleware in ``main.py`` returns
HTTP 202 ("preparing private lane") for ``/api/mesh/*`` requests
when the Wormhole supervisor is not yet at the required transport
tier. In tests that's always — Wormhole is not running. Patching
``_minimum_transport_tier`` to return None disables the tier check
for the duration of the test, letting the request reach the route
(and therefore reach the ``Depends(require_admin)`` we are testing).
"""
import main
with patch("main._minimum_transport_tier", return_value=None):
yield TestClient(main.app, raise_server_exceptions=False)
@pytest.fixture
def mock_ledger():
"""Replace oracle_ledger methods so tests don't mutate persistent state.
The handler does ``from services.mesh.mesh_oracle import oracle_ledger``
at call time, so we patch the module attribute.
"""
fake = MagicMock()
fake.resolve_market.return_value = (0, 0)
fake.resolve_market_stakes.return_value = {"winners": 0, "losers": 0}
fake.resolve_expired_stakes.return_value = []
with patch("services.mesh.mesh_oracle.oracle_ledger", fake):
yield fake
# ---------------------------------------------------------------------------
# /api/mesh/oracle/resolve — issue #240
# ---------------------------------------------------------------------------
class TestOracleResolveAuthGate:
def test_anonymous_caller_is_rejected(self, client, mock_ledger):
with patch("auth._current_admin_key", return_value=_ADMIN_KEY):
r = client.post(
"/api/mesh/oracle/resolve",
json={"market_title": "test-market", "outcome": "Yes"},
)
assert r.status_code == 403
# Critically: the ledger mutator must NOT have been called on a 403.
assert mock_ledger.resolve_market.call_count == 0
assert mock_ledger.resolve_market_stakes.call_count == 0
def test_wrong_admin_key_rejected(self, client, mock_ledger):
with patch("auth._current_admin_key", return_value=_ADMIN_KEY):
r = client.post(
"/api/mesh/oracle/resolve",
headers={"X-Admin-Key": "this-key-is-wrong"},
json={"market_title": "test-market", "outcome": "Yes"},
)
assert r.status_code == 403
assert mock_ledger.resolve_market.call_count == 0
def test_valid_admin_key_passes_auth_gate(self, client, mock_ledger):
with patch("auth._current_admin_key", return_value=_ADMIN_KEY):
r = client.post(
"/api/mesh/oracle/resolve",
headers={"X-Admin-Key": _ADMIN_KEY},
json={"market_title": "test-market", "outcome": "Yes"},
)
# The auth gate let us through. The handler ran and called the
# (mocked) ledger.
assert r.status_code == 200
assert mock_ledger.resolve_market.call_count == 1
assert mock_ledger.resolve_market.call_args[0] == ("test-market", "Yes")
def test_admin_key_unset_blocks_in_production_posture(self, client, mock_ledger):
"""When ADMIN_KEY env is not configured at all and we're not in
debug, the endpoint must still refuse never silently accept."""
with (
patch("auth._current_admin_key", return_value=""),
patch("auth._allow_insecure_admin", return_value=False),
patch("auth._debug_mode_enabled", return_value=False),
patch("auth._scoped_admin_tokens", return_value={}),
):
r = client.post(
"/api/mesh/oracle/resolve",
json={"market_title": "test-market", "outcome": "Yes"},
)
assert r.status_code == 403
assert mock_ledger.resolve_market.call_count == 0
# ---------------------------------------------------------------------------
# /api/mesh/oracle/resolve-stakes — issue #241
# ---------------------------------------------------------------------------
class TestOracleResolveStakesAuthGate:
def test_anonymous_caller_is_rejected(self, client, mock_ledger):
with patch("auth._current_admin_key", return_value=_ADMIN_KEY):
r = client.post("/api/mesh/oracle/resolve-stakes")
assert r.status_code == 403
assert mock_ledger.resolve_expired_stakes.call_count == 0
def test_wrong_admin_key_rejected(self, client, mock_ledger):
with patch("auth._current_admin_key", return_value=_ADMIN_KEY):
r = client.post(
"/api/mesh/oracle/resolve-stakes",
headers={"X-Admin-Key": "nope"},
)
assert r.status_code == 403
assert mock_ledger.resolve_expired_stakes.call_count == 0
def test_valid_admin_key_passes_auth_gate(self, client, mock_ledger):
with patch("auth._current_admin_key", return_value=_ADMIN_KEY):
r = client.post(
"/api/mesh/oracle/resolve-stakes",
headers={"X-Admin-Key": _ADMIN_KEY},
)
assert r.status_code == 200
assert mock_ledger.resolve_expired_stakes.call_count == 1
body = r.json()
assert body["ok"] is True
assert body["count"] == 0
def test_admin_key_unset_blocks_in_production_posture(self, client, mock_ledger):
with (
patch("auth._current_admin_key", return_value=""),
patch("auth._allow_insecure_admin", return_value=False),
patch("auth._debug_mode_enabled", return_value=False),
patch("auth._scoped_admin_tokens", return_value={}),
):
r = client.post("/api/mesh/oracle/resolve-stakes")
assert r.status_code == 403
assert mock_ledger.resolve_expired_stakes.call_count == 0
+19 -3
View File
@@ -87,16 +87,32 @@ class TestRequireLocalOperator:
assert self._call_with_host("172.16.0.5") == 403
def test_docker_bridge_blocked_without_compose_opt_in(self):
# Even if DNS would resolve the frontend hostname to this IP,
# the env opt-in is required.
with patch.dict("os.environ", {"SHADOWBROKER_TRUST_DOCKER_BRIDGE_LOCAL_OPERATOR": ""}):
assert self._call_with_host("172.18.0.3") == 403
with patch("auth._resolve_trusted_bridge_ips", return_value=frozenset({"172.18.0.3"})):
assert self._call_with_host("172.18.0.3") == 403
def test_docker_bridge_passes_with_compose_opt_in(self):
# Issue #250: opt-in alone is no longer sufficient — the source IP
# must also reverse-match a trusted frontend container hostname.
# Here we simulate Docker DNS resolving "frontend" to 172.18.0.3.
with patch.dict("os.environ", {"SHADOWBROKER_TRUST_DOCKER_BRIDGE_LOCAL_OPERATOR": "1"}):
assert self._call_with_host("172.18.0.3") == 200
with patch("auth._resolve_trusted_bridge_ips", return_value=frozenset({"172.18.0.3"})):
assert self._call_with_host("172.18.0.3") == 200
def test_unknown_bridge_ip_blocked_even_with_compose_opt_in(self):
# Issue #250 core regression: a rogue container on the same bridge
# whose IP is NOT in the resolved frontend hostname set must NOT
# be trusted, even when the bridge opt-in flag is on.
with patch.dict("os.environ", {"SHADOWBROKER_TRUST_DOCKER_BRIDGE_LOCAL_OPERATOR": "1"}):
with patch("auth._resolve_trusted_bridge_ips", return_value=frozenset({"172.18.0.3"})):
assert self._call_with_host("172.18.0.99") == 403
def test_lan_ip_still_blocked_with_compose_opt_in(self):
with patch.dict("os.environ", {"SHADOWBROKER_TRUST_DOCKER_BRIDGE_LOCAL_OPERATOR": "1"}):
assert self._call_with_host("192.168.1.100") == 403
with patch("auth._resolve_trusted_bridge_ips", return_value=frozenset({"172.18.0.3"})):
assert self._call_with_host("192.168.1.100") == 403
def test_rfc1918_192168_blocked_without_key(self):
assert self._call_with_host("192.168.1.100") == 403
@@ -0,0 +1,366 @@
"""Issue #256 (tg12): per-peer HMAC secrets must defeat cross-peer
impersonation.
Before the fix, ALL peer-push HMACs were derived from the single
fleet-shared ``MESH_PEER_PUSH_SECRET``. The receiver could only prove
"this request was signed by someone who knows the fleet secret" not
which peer signed it. Any peer that knew the secret could compute the
expected HMAC for any other peer's URL and impersonate that peer.
The fix introduces ``MESH_PEER_SECRETS``, a per-peer URL-to-secret map.
When a peer URL appears there:
- Only the listed per-peer secret is accepted for that URL.
- The global ``MESH_PEER_PUSH_SECRET`` is ignored for that specific URL.
- A peer that knows only the global secret (or a different peer's
per-peer secret) cannot forge a request claiming to be that peer.
When a peer URL is NOT listed (the common case for single-peer installs
and for migration windows), the resolver falls back to the global
secret preserving existing behavior with zero operator action.
These tests exercise ``resolve_peer_key_for_url`` directly so we cover
the security contract without spinning up a full mesh node.
"""
from __future__ import annotations
import hashlib
import hmac
import pytest
# ---------------------------------------------------------------------------
# _lookup_per_peer_secret — env parsing
# ---------------------------------------------------------------------------
class TestLookupPerPeerSecret:
def setup_method(self):
# Invalidate the parser cache so each test sees its own env state.
from services.mesh import mesh_crypto
mesh_crypto._PEER_SECRETS_CACHE = {}
mesh_crypto._PEER_SECRETS_CACHE_RAW = ""
def test_returns_empty_when_env_unset(self, monkeypatch):
from services.mesh.mesh_crypto import _lookup_per_peer_secret
monkeypatch.delenv("MESH_PEER_SECRETS", raising=False)
assert _lookup_per_peer_secret("https://peer.example") == ""
def test_returns_empty_when_env_blank(self, monkeypatch):
from services.mesh.mesh_crypto import _lookup_per_peer_secret
monkeypatch.setenv("MESH_PEER_SECRETS", "")
assert _lookup_per_peer_secret("https://peer.example") == ""
def test_returns_per_peer_secret_for_listed_url(self, monkeypatch):
from services.mesh.mesh_crypto import _lookup_per_peer_secret
monkeypatch.setenv(
"MESH_PEER_SECRETS",
"https://peer-a.example=secretA,https://peer-b.example=secretB",
)
assert _lookup_per_peer_secret("https://peer-a.example") == "secretA"
assert _lookup_per_peer_secret("https://peer-b.example") == "secretB"
def test_returns_empty_for_url_not_listed(self, monkeypatch):
from services.mesh.mesh_crypto import _lookup_per_peer_secret
monkeypatch.setenv(
"MESH_PEER_SECRETS",
"https://peer-a.example=secretA",
)
assert _lookup_per_peer_secret("https://other.example") == ""
def test_url_is_normalized_before_lookup(self, monkeypatch):
from services.mesh.mesh_crypto import _lookup_per_peer_secret
# Configure with a trailing slash + uppercase host. Lookup with
# plain lowercase host. Both should normalize to the same key.
monkeypatch.setenv(
"MESH_PEER_SECRETS",
"https://Peer-A.Example/=secretA",
)
assert _lookup_per_peer_secret("https://peer-a.example") == "secretA"
def test_whitespace_around_entries_is_stripped(self, monkeypatch):
from services.mesh.mesh_crypto import _lookup_per_peer_secret
monkeypatch.setenv(
"MESH_PEER_SECRETS",
" https://peer-a.example = secretA , https://peer-b.example=secretB ",
)
assert _lookup_per_peer_secret("https://peer-a.example") == "secretA"
assert _lookup_per_peer_secret("https://peer-b.example") == "secretB"
def test_malformed_entries_are_skipped_not_raised(self, monkeypatch):
"""A garbled MESH_PEER_SECRETS value must NOT crash the resolver.
Bad entries are silently dropped; well-formed entries still work.
This is the "fail-forward, not loud" rule a typo in operator
config should not take the whole backend down."""
from services.mesh.mesh_crypto import _lookup_per_peer_secret
monkeypatch.setenv(
"MESH_PEER_SECRETS",
"no_equals_sign,=missing_url,https://no.secret=,https://good.example=secretGood",
)
assert _lookup_per_peer_secret("https://good.example") == "secretGood"
# The malformed ones produce no entry (and don't poison the cache).
assert _lookup_per_peer_secret("https://no.secret") == ""
def test_cache_invalidates_on_env_change(self, monkeypatch):
"""A test (or operator) updating MESH_PEER_SECRETS must see the
new value immediately no process restart required."""
from services.mesh.mesh_crypto import _lookup_per_peer_secret
monkeypatch.setenv("MESH_PEER_SECRETS", "https://a.example=first")
assert _lookup_per_peer_secret("https://a.example") == "first"
monkeypatch.setenv("MESH_PEER_SECRETS", "https://a.example=second")
assert _lookup_per_peer_secret("https://a.example") == "second"
# ---------------------------------------------------------------------------
# resolve_peer_key_for_url — precedence + fallback
# ---------------------------------------------------------------------------
class TestResolvePeerKeyForUrl:
def setup_method(self):
from services.mesh import mesh_crypto
mesh_crypto._PEER_SECRETS_CACHE = {}
mesh_crypto._PEER_SECRETS_CACHE_RAW = ""
def _fake_settings(self, global_secret: str):
from unittest.mock import MagicMock
s = MagicMock()
s.MESH_PEER_PUSH_SECRET = global_secret
return s
def test_falls_back_to_global_when_no_per_peer_entry(self, monkeypatch):
"""Single-peer installs: MESH_PEER_SECRETS empty, MESH_PEER_PUSH_SECRET
set must keep working as before."""
from services.mesh.mesh_crypto import (
resolve_peer_key_for_url,
_derive_peer_key,
)
monkeypatch.delenv("MESH_PEER_SECRETS", raising=False)
with monkeypatch.context() as m:
m.setattr(
"services.config.get_settings",
lambda: self._fake_settings("global-secret"),
)
key = resolve_peer_key_for_url("https://peer.example")
expected = _derive_peer_key("global-secret", "https://peer.example")
assert key == expected
assert len(key) == 32 # SHA-256 output
def test_per_peer_secret_takes_precedence_over_global(self, monkeypatch):
from services.mesh.mesh_crypto import (
resolve_peer_key_for_url,
_derive_peer_key,
)
monkeypatch.setenv(
"MESH_PEER_SECRETS",
"https://peer-a.example=per-peer-a-secret",
)
with monkeypatch.context() as m:
m.setattr(
"services.config.get_settings",
lambda: self._fake_settings("global-secret"),
)
key = resolve_peer_key_for_url("https://peer-a.example")
expected_per_peer = _derive_peer_key(
"per-peer-a-secret", "https://peer-a.example"
)
expected_global = _derive_peer_key("global-secret", "https://peer-a.example")
assert key == expected_per_peer
assert key != expected_global
def test_unlisted_peer_uses_global_during_migration(self, monkeypatch):
"""Partial migration: peer A is in MESH_PEER_SECRETS, peer B is
not yet. Peer B must keep working under the global secret."""
from services.mesh.mesh_crypto import (
resolve_peer_key_for_url,
_derive_peer_key,
)
monkeypatch.setenv(
"MESH_PEER_SECRETS",
"https://peer-a.example=per-peer-a-secret",
)
with monkeypatch.context() as m:
m.setattr(
"services.config.get_settings",
lambda: self._fake_settings("global-secret"),
)
key_a = resolve_peer_key_for_url("https://peer-a.example")
key_b = resolve_peer_key_for_url("https://peer-b.example")
expected_b = _derive_peer_key("global-secret", "https://peer-b.example")
assert key_b == expected_b
# Peer A's per-peer key must differ from peer B's global key
# (they're keyed by different secrets and different URLs).
assert key_a != key_b
def test_returns_empty_when_no_secret_available(self, monkeypatch):
from services.mesh.mesh_crypto import resolve_peer_key_for_url
monkeypatch.delenv("MESH_PEER_SECRETS", raising=False)
with monkeypatch.context() as m:
m.setattr(
"services.config.get_settings",
lambda: self._fake_settings(""),
)
key = resolve_peer_key_for_url("https://peer.example")
assert key == b""
def test_returns_empty_when_url_is_unparseable(self, monkeypatch):
from services.mesh.mesh_crypto import resolve_peer_key_for_url
with monkeypatch.context() as m:
m.setattr(
"services.config.get_settings",
lambda: self._fake_settings("global-secret"),
)
assert resolve_peer_key_for_url("") == b""
assert resolve_peer_key_for_url("not-a-url") == b""
assert resolve_peer_key_for_url(None) == b""
# ---------------------------------------------------------------------------
# The actual #256 attack: peer A cannot impersonate peer B
# ---------------------------------------------------------------------------
class TestCrossPeerImpersonationRefused:
"""The core regression: when MESH_PEER_SECRETS is configured, a peer
that knows ONLY the global secret (or a different peer's per-peer
secret) cannot produce a valid HMAC for another peer's URL."""
def setup_method(self):
from services.mesh import mesh_crypto
mesh_crypto._PEER_SECRETS_CACHE = {}
mesh_crypto._PEER_SECRETS_CACHE_RAW = ""
def _hmac(self, key: bytes, body: bytes) -> str:
return hmac.new(key, body, hashlib.sha256).hexdigest()
def test_peer_a_global_secret_cannot_forge_peer_b_hmac(self, monkeypatch):
from services.mesh.mesh_crypto import (
resolve_peer_key_for_url,
_derive_peer_key,
)
from unittest.mock import MagicMock
# Receiver has BOTH the global secret AND a per-peer secret for B.
monkeypatch.setenv(
"MESH_PEER_SECRETS",
"https://peer-b.example=per-peer-b-secret",
)
settings = MagicMock()
settings.MESH_PEER_PUSH_SECRET = "global-secret"
monkeypatch.setattr(
"services.config.get_settings", lambda: settings
)
body = b'{"events": [{"id": 1}]}'
# Attacker (peer A) knows only the global secret. Tries to forge
# an HMAC claiming to be peer B.
attacker_key = _derive_peer_key("global-secret", "https://peer-b.example")
attacker_hmac = self._hmac(attacker_key, body)
# Receiver derives B's expected key from B's per-peer secret.
receiver_key = resolve_peer_key_for_url("https://peer-b.example")
expected_hmac = self._hmac(receiver_key, body)
# The forgery MUST NOT match.
assert attacker_hmac != expected_hmac
def test_peer_a_per_peer_secret_cannot_forge_peer_b_hmac(self, monkeypatch):
"""Even harder case: peer A has its OWN per-peer secret, but
still does not know peer B's per-peer secret, and so cannot
forge an HMAC for peer B."""
from services.mesh.mesh_crypto import (
resolve_peer_key_for_url,
_derive_peer_key,
)
from unittest.mock import MagicMock
monkeypatch.setenv(
"MESH_PEER_SECRETS",
"https://peer-a.example=secretA,https://peer-b.example=secretB",
)
settings = MagicMock()
settings.MESH_PEER_PUSH_SECRET = ""
monkeypatch.setattr(
"services.config.get_settings", lambda: settings
)
body = b'{"events": [{"id": 99}]}'
# Attacker A tries to forge for B using its own secret (secretA).
attacker_key = _derive_peer_key("secretA", "https://peer-b.example")
attacker_hmac = self._hmac(attacker_key, body)
receiver_key = resolve_peer_key_for_url("https://peer-b.example")
expected_hmac = self._hmac(receiver_key, body)
assert attacker_hmac != expected_hmac
def test_legitimate_peer_b_request_verifies(self, monkeypatch):
"""Positive control: when peer B uses ITS per-peer secret and
claims to be itself, the receiver accepts the HMAC."""
from services.mesh.mesh_crypto import resolve_peer_key_for_url
from unittest.mock import MagicMock
monkeypatch.setenv(
"MESH_PEER_SECRETS",
"https://peer-b.example=secretB",
)
settings = MagicMock()
settings.MESH_PEER_PUSH_SECRET = ""
monkeypatch.setattr(
"services.config.get_settings", lambda: settings
)
body = b'{"events": [{"id": 7}]}'
# Peer B and the receiver both call resolve_peer_key_for_url.
sender_key = resolve_peer_key_for_url("https://peer-b.example")
receiver_key = resolve_peer_key_for_url("https://peer-b.example")
sender_hmac = self._hmac(sender_key, body)
expected_hmac = self._hmac(receiver_key, body)
assert sender_hmac == expected_hmac
def test_single_peer_install_zero_behavior_change(self, monkeypatch):
"""The "no UX hostility" guarantee: an install with the global
secret set and NO MESH_PEER_SECRETS entries must derive exactly
the same key as before this change."""
from services.mesh.mesh_crypto import (
resolve_peer_key_for_url,
_derive_peer_key,
)
from unittest.mock import MagicMock
monkeypatch.delenv("MESH_PEER_SECRETS", raising=False)
settings = MagicMock()
settings.MESH_PEER_PUSH_SECRET = "legacy-global-secret"
monkeypatch.setattr(
"services.config.get_settings", lambda: settings
)
# The legacy derivation that every prior call site used.
legacy_key = _derive_peer_key("legacy-global-secret", "https://peer.example")
# The new resolver, with no per-peer entries configured.
new_key = resolve_peer_key_for_url("https://peer.example")
assert new_key == legacy_key
@@ -0,0 +1,91 @@
"""Issues #218 / #219 (tg12): outbound Wikipedia + Wikidata calls must
identify ShadowBroker via the Wikimedia-recommended User-Agent /
Api-User-Agent headers.
Before this fix, ``backend/services/region_dossier.py`` called
``fetch_with_curl(url)`` with no explicit headers, falling back to the
generic project default UA. That sent a too-anonymous identifier to
Wikimedia. Per Wikimedia's policy
(https://foundation.wikimedia.org/wiki/Policy:Wikimedia_Foundation_User-Agent_Policy)
the API caller should send a stable, contactable identifier so Wikimedia
operators can rate-limit or reach the project.
This test does NOT make network calls. It patches ``fetch_with_curl``
and asserts the headers that get passed through.
"""
from __future__ import annotations
from unittest.mock import MagicMock, patch
import pytest
def _fake_resp(payload: dict, status: int = 200) -> MagicMock:
r = MagicMock()
r.status_code = status
r.json.return_value = payload
return r
def test_wikidata_call_passes_wikimedia_request_headers():
from services import region_dossier
calls = []
def fake_fetch(url, **kwargs):
calls.append(kwargs.get("headers"))
return _fake_resp({"results": {"bindings": []}})
with patch.object(region_dossier, "fetch_with_curl", side_effect=fake_fetch):
region_dossier._fetch_wikidata_leader("Testlandia")
assert calls, "fetch_with_curl was not called"
headers = calls[0] or {}
assert "User-Agent" in headers
assert "Api-User-Agent" in headers
# Stable identifier should mention the project + a contact path.
assert "Shadowbroker" in headers["Api-User-Agent"] or "ShadowBroker" in headers["Api-User-Agent"]
assert "github.com" in headers["Api-User-Agent"].lower()
def test_wikipedia_summary_call_passes_wikimedia_request_headers():
from services import region_dossier
calls = []
def fake_fetch(url, **kwargs):
calls.append((url, kwargs.get("headers")))
return _fake_resp(
{
"type": "standard",
"description": "test desc",
"extract": "test extract",
"thumbnail": {"source": ""},
}
)
with patch.object(region_dossier, "fetch_with_curl", side_effect=fake_fetch):
region_dossier._fetch_local_wiki_summary("Paris", "France")
# At least one Wikipedia REST call was issued.
wikipedia_calls = [c for c in calls if "wikipedia.org" in c[0]]
assert wikipedia_calls, "no Wikipedia call was issued"
for url, headers in wikipedia_calls:
headers = headers or {}
assert "User-Agent" in headers, f"missing User-Agent on {url}"
assert "Api-User-Agent" in headers, f"missing Api-User-Agent on {url}"
assert "github.com" in headers["Api-User-Agent"].lower()
def test_wikimedia_headers_constant_is_stable():
"""Regression guard: if someone removes the contact path from the
Api-User-Agent we want a loud test failure, not a silent ToS drift.
"""
from services.region_dossier import _WIKIMEDIA_REQUEST_HEADERS
aua = _WIKIMEDIA_REQUEST_HEADERS.get("Api-User-Agent", "")
assert "Shadowbroker" in aua or "ShadowBroker" in aua
assert "github.com" in aua.lower()
# Must include a path Wikimedia operators can use to contact us
# (we use /issues against the public repo).
assert "issues" in aua.lower()
@@ -0,0 +1,263 @@
"""Issues #243, #252, #253 (tg12): settings endpoints must not leak
operational posture to unauthenticated callers.
- **#243**: ``GET /api/settings/wormhole``, ``/api/settings/privacy-profile``,
and ``/api/settings/node`` were leaking transport choice, anonymous-mode
state, the named privacy profile, and node-participant state to any
unauthenticated caller. The fix tightens the redaction allowlists to
expose ONLY a bare "is this feature on?" boolean and gates node mode
behind authenticated reads.
- **#252**: ``GET /api/settings/news-feeds`` returned the operator's full
curated feed inventory (names + URLs) to anyone. Now gated on
local-operator.
- **#253**: ``GET /api/settings/timemachine`` returned whether archival
capture is enabled to anyone. Now gated on local-operator.
Auth model: ``require_local_operator`` allows loopback (Tauri shell),
the Docker bridge frontend container (via the hostname-bound trust from
PR #278), and any caller that presents the configured admin key.
Anonymous LAN or internet callers do NOT pass and either receive 403
(news-feeds, timemachine) or a redacted minimum (wormhole / node).
"""
from __future__ import annotations
from unittest.mock import patch, MagicMock
import pytest
from fastapi.testclient import TestClient
_ADMIN_KEY = "test-admin-key-for-round5-fixture-32+chars"
@pytest.fixture
def client():
"""TestClient with the private-lane transport middleware disabled.
Same shape as the oracle resolve fixture the mesh privacy
middleware returns 202 for ``/api/settings/*`` under TestClient
because Wormhole is not actually running. Patching out the tier
requirement lets requests reach the route's auth gate.
"""
import main
with patch("main._minimum_transport_tier", return_value=None):
yield TestClient(main.app, raise_server_exceptions=False)
# ---------------------------------------------------------------------------
# #243: Wormhole posture redaction
# ---------------------------------------------------------------------------
class TestWormholeSettingsRedaction:
"""``GET /api/settings/wormhole`` must NOT leak transport choice or
anonymous-mode state to unauthenticated callers."""
def _read_settings_payload(self):
return {
"enabled": True,
"transport": "tor_arti",
"anonymous_mode": True,
"privacy_profile": "high",
"socks_proxy": "socks5h://127.0.0.1:9050",
}
def test_anonymous_caller_sees_only_enabled_bool(self, client):
with (
patch("main.read_wormhole_settings", return_value=self._read_settings_payload()),
patch("routers.wormhole.read_wormhole_settings", return_value=self._read_settings_payload()),
patch("services.wormhole_settings.read_wormhole_settings", return_value=self._read_settings_payload()),
patch("auth._current_admin_key", return_value=_ADMIN_KEY),
):
r = client.get("/api/settings/wormhole")
assert r.status_code == 200
body = r.json()
# Only the bare "is Wormhole on?" boolean is exposed publicly.
assert "enabled" in body
assert body["enabled"] is True
# Posture fields the audit flagged must be absent.
assert "transport" not in body
assert "anonymous_mode" not in body
assert "privacy_profile" not in body
assert "socks_proxy" not in body
def test_authenticated_caller_sees_full_state(self, client):
with (
patch("main.read_wormhole_settings", return_value=self._read_settings_payload()),
patch("routers.wormhole.read_wormhole_settings", return_value=self._read_settings_payload()),
patch("services.wormhole_settings.read_wormhole_settings", return_value=self._read_settings_payload()),
patch("auth._current_admin_key", return_value=_ADMIN_KEY),
):
r = client.get(
"/api/settings/wormhole",
headers={"X-Admin-Key": _ADMIN_KEY},
)
assert r.status_code == 200
body = r.json()
# All fields visible when authenticated.
assert body["enabled"] is True
assert body["transport"] == "tor_arti"
assert body["anonymous_mode"] is True
assert body["privacy_profile"] == "high"
class TestPrivacyProfileRedaction:
"""``GET /api/settings/privacy-profile`` must NOT leak the named
profile to unauthenticated callers (the profile name itself
discloses operator intent)."""
def _payload(self):
return {
"enabled": True,
"transport": "tor_arti",
"anonymous_mode": True,
"privacy_profile": "high",
}
def test_anonymous_caller_sees_only_wormhole_enabled_bool(self, client):
with (
patch("main.read_wormhole_settings", return_value=self._payload()),
patch("routers.wormhole.read_wormhole_settings", return_value=self._payload()),
patch("services.wormhole_settings.read_wormhole_settings", return_value=self._payload()),
patch("auth._current_admin_key", return_value=_ADMIN_KEY),
):
r = client.get("/api/settings/privacy-profile")
assert r.status_code == 200
body = r.json()
assert "wormhole_enabled" in body
assert body["wormhole_enabled"] is True
# The named profile, transport, and anonymous mode must NOT
# leak to anonymous callers.
assert "profile" not in body or body.get("profile") is None
assert "transport" not in body
assert "anonymous_mode" not in body
def test_authenticated_caller_sees_named_profile_and_transport(self, client):
with (
patch("main.read_wormhole_settings", return_value=self._payload()),
patch("routers.wormhole.read_wormhole_settings", return_value=self._payload()),
patch("services.wormhole_settings.read_wormhole_settings", return_value=self._payload()),
patch("auth._current_admin_key", return_value=_ADMIN_KEY),
):
r = client.get(
"/api/settings/privacy-profile",
headers={"X-Admin-Key": _ADMIN_KEY},
)
assert r.status_code == 200
body = r.json()
assert body["profile"] == "high"
assert body["wormhole_enabled"] is True
assert body["transport"] == "tor_arti"
assert body["anonymous_mode"] is True
class TestNodeSettingsRedaction:
"""``GET /api/settings/node`` must NOT disclose node_mode or
node_enabled to anonymous callers."""
def _node_data(self):
return {"some_node_field": "value"}
def test_anonymous_caller_sees_empty_stub(self, client):
with (
patch("services.node_settings.read_node_settings", return_value=self._node_data()),
patch("routers.admin._current_node_mode", return_value="participant"),
patch("routers.admin._participant_node_enabled", return_value=True),
patch("auth._current_admin_key", return_value=_ADMIN_KEY),
):
r = client.get("/api/settings/node")
assert r.status_code == 200
body = r.json()
# No posture fields.
assert "node_mode" not in body
assert "node_enabled" not in body
assert "some_node_field" not in body
def test_authenticated_caller_sees_full_node_state(self, client):
with (
patch("services.node_settings.read_node_settings", return_value=self._node_data()),
patch("routers.admin._current_node_mode", return_value="participant"),
patch("routers.admin._participant_node_enabled", return_value=True),
patch("auth._current_admin_key", return_value=_ADMIN_KEY),
):
r = client.get(
"/api/settings/node",
headers={"X-Admin-Key": _ADMIN_KEY},
)
assert r.status_code == 200
body = r.json()
assert body["node_mode"] == "participant"
assert body["node_enabled"] is True
assert body["some_node_field"] == "value"
# ---------------------------------------------------------------------------
# #252: news-feeds auth gate
# ---------------------------------------------------------------------------
class TestNewsFeedsAuthGate:
def _fake_feeds(self):
return [
{"name": "Custom Internal", "url": "https://internal.example/rss", "weight": 5},
{"name": "Default News", "url": "https://news.example/rss", "weight": 3},
]
def test_anonymous_caller_rejected(self, client):
with (
patch("services.news_feed_config.get_feeds", return_value=self._fake_feeds()) as get_feeds,
patch("auth._current_admin_key", return_value=_ADMIN_KEY),
):
r = client.get("/api/settings/news-feeds")
assert r.status_code == 403
# Critically: the underlying config read must NOT have been performed
# (else the response body could leak the count via response timing).
assert get_feeds.call_count == 0
def test_authenticated_caller_sees_full_feed_inventory(self, client):
with (
patch("services.news_feed_config.get_feeds", return_value=self._fake_feeds()),
patch("auth._current_admin_key", return_value=_ADMIN_KEY),
):
r = client.get(
"/api/settings/news-feeds",
headers={"X-Admin-Key": _ADMIN_KEY},
)
assert r.status_code == 200
body = r.json()
assert len(body) == 2
assert body[0]["name"] == "Custom Internal"
assert body[0]["url"] == "https://internal.example/rss"
# ---------------------------------------------------------------------------
# #253: timemachine auth gate
# ---------------------------------------------------------------------------
class TestTimemachineAuthGate:
def test_anonymous_caller_rejected(self, client):
node_data = {"timemachine_enabled": True}
with (
patch("services.node_settings.read_node_settings", return_value=node_data),
patch("auth._current_admin_key", return_value=_ADMIN_KEY),
):
r = client.get("/api/settings/timemachine")
assert r.status_code == 403
def test_authenticated_caller_sees_enabled_state(self, client):
node_data = {"timemachine_enabled": True}
with (
patch("services.node_settings.read_node_settings", return_value=node_data),
patch("auth._current_admin_key", return_value=_ADMIN_KEY),
):
r = client.get(
"/api/settings/timemachine",
headers={"X-Admin-Key": _ADMIN_KEY},
)
assert r.status_code == 200
body = r.json()
assert body["enabled"] is True
assert "storage_warning" in body
@@ -0,0 +1,222 @@
"""Issue #251 (tg12): Tor bundle extraction must refuse symlink and
hardlink members.
The previous extractor checked ``member.name`` against path traversal
but never inspected ``member.linkname``. Python 3.11's ``tarfile``
honors symlinks during ``extractall()``, so a malicious archive could
ship a member named ``innocent.txt`` whose linkname points at an
arbitrary filesystem location. After extraction, reads of innocent.txt
dereference to that location; writes corrupt it.
The fix categorically refuses any link member during extraction.
Tor Expert Bundles never legitimately contain symlinks or hardlinks,
so this is non-disruptive for real updates and a hard stop for hostile
archives.
These tests build synthetic tar archives covering each refused case
and assert ``_extract_tor_bundle_safely`` rejects them.
"""
import io
import os
import stat
import tarfile
from pathlib import Path
import pytest
from services.tor_hidden_service import _extract_tor_bundle_safely
def _build_archive(tmp_path: Path, members: list) -> Path:
"""Write a .tar.gz with the given (name, builder) pairs.
Each builder is called with the open tarfile and is responsible for
adding its member however it likes (regular file, symlink, etc.).
"""
archive = tmp_path / "test_bundle.tar.gz"
with tarfile.open(str(archive), "w:gz") as tar:
for name, builder in members:
builder(tar, name)
return archive
def _add_regular_file(tar: tarfile.TarFile, name: str, payload: bytes = b"hello") -> None:
info = tarfile.TarInfo(name)
info.size = len(payload)
info.mode = 0o644
info.type = tarfile.REGTYPE
tar.addfile(info, io.BytesIO(payload))
def _add_symlink(tar: tarfile.TarFile, name: str, linkname: str) -> None:
info = tarfile.TarInfo(name)
info.size = 0
info.type = tarfile.SYMTYPE
info.linkname = linkname
info.mode = 0o777
tar.addfile(info)
def _add_hardlink(tar: tarfile.TarFile, name: str, linkname: str) -> None:
info = tarfile.TarInfo(name)
info.size = 0
info.type = tarfile.LNKTYPE
info.linkname = linkname
info.mode = 0o644
tar.addfile(info)
def _add_fifo(tar: tarfile.TarFile, name: str) -> None:
info = tarfile.TarInfo(name)
info.type = tarfile.FIFOTYPE
info.mode = 0o644
tar.addfile(info)
def test_clean_archive_extracts_successfully(tmp_path):
"""A normal archive with only regular files extracts fine."""
install_dir = tmp_path / "install"
install_dir.mkdir()
def add_normal(tar, name):
_add_regular_file(tar, name, b"clean content")
archive = _build_archive(
tmp_path,
[
("tor/tor.exe", add_normal),
("tor/data/geoip", add_normal),
],
)
assert _extract_tor_bundle_safely(archive, install_dir) is True
assert (install_dir / "tor" / "tor.exe").is_file()
assert (install_dir / "tor" / "data" / "geoip").is_file()
def test_symlink_member_is_rejected(tmp_path, caplog):
"""Issue #251 core regression: symlink members are refused."""
install_dir = tmp_path / "install"
install_dir.mkdir()
archive = _build_archive(
tmp_path,
[
("tor/innocent.txt", lambda t, n: _add_symlink(t, n, "/etc/passwd")),
],
)
import logging
with caplog.at_level(logging.ERROR):
result = _extract_tor_bundle_safely(archive, install_dir)
assert result is False
# No file should have been created
assert not (install_dir / "tor" / "innocent.txt").exists()
# Log should explain why
assert any(
"symlinks/hardlinks are not allowed" in rec.getMessage()
for rec in caplog.records
)
def test_hardlink_member_is_rejected(tmp_path):
"""Hardlinks are refused for the same reason as symlinks."""
install_dir = tmp_path / "install"
install_dir.mkdir()
archive = _build_archive(
tmp_path,
[
("tor/regular.txt", lambda t, n: _add_regular_file(t, n)),
("tor/sneaky.txt", lambda t, n: _add_hardlink(t, n, "regular.txt")),
],
)
assert _extract_tor_bundle_safely(archive, install_dir) is False
# The whole extraction is refused even though only one member is bad.
assert not (install_dir / "tor" / "regular.txt").exists()
def test_symlink_with_relative_target_still_rejected(tmp_path):
"""Even a relative symlink target inside the install dir is refused.
We don't allow symlinks at all — there is no legitimate Tor bundle
use case for them, and an attacker can chain link redirections in
ways the path-resolution check is poor at catching.
"""
install_dir = tmp_path / "install"
install_dir.mkdir()
archive = _build_archive(
tmp_path,
[
("tor/alias.txt", lambda t, n: _add_symlink(t, n, "tor/tor.exe")),
],
)
assert _extract_tor_bundle_safely(archive, install_dir) is False
def test_fifo_or_device_member_is_rejected(tmp_path):
"""Non-regular-non-directory members (FIFOs, devices) are refused."""
install_dir = tmp_path / "install"
install_dir.mkdir()
archive = _build_archive(
tmp_path,
[
("tor/weird.fifo", _add_fifo),
],
)
assert _extract_tor_bundle_safely(archive, install_dir) is False
def test_path_traversal_member_is_rejected(tmp_path):
"""Pre-existing path-traversal guard still works under the new shape."""
install_dir = tmp_path / "install"
install_dir.mkdir()
def add_traversal(tar, name):
_add_regular_file(tar, name)
# ../../escape.txt resolves outside install_dir on most platforms.
archive = _build_archive(
tmp_path,
[
("../../escape.txt", add_traversal),
],
)
assert _extract_tor_bundle_safely(archive, install_dir) is False
def test_malformed_tar_is_rejected(tmp_path):
"""A corrupt/non-tar file is rejected without crashing."""
install_dir = tmp_path / "install"
install_dir.mkdir()
bogus = tmp_path / "not-a-tar.tar.gz"
bogus.write_bytes(b"this is not a tar archive at all")
assert _extract_tor_bundle_safely(bogus, install_dir) is False
def test_extraction_failure_does_not_leave_partial_state_referenced_to_caller(tmp_path):
"""When extraction fails partway, the caller relies on a False return
to know it must clean up. We test the contract here actual cleanup
of files that may have been written by tar.extractall() before the
failure point isn't part of THIS helper's responsibility (the caller
deletes the install dir if needed)."""
install_dir = tmp_path / "install"
install_dir.mkdir()
# Hostile archive: one good file, then a symlink. Whether the good
# file was written or not, the return value must be False so the
# caller refuses the bundle.
archive = _build_archive(
tmp_path,
[
("tor/clean.txt", lambda t, n: _add_regular_file(t, n)),
("tor/evil-link.txt", lambda t, n: _add_symlink(t, n, "/etc/passwd")),
],
)
assert _extract_tor_bundle_safely(archive, install_dir) is False
@@ -0,0 +1,338 @@
"""Issue #231 — self-update SHA-256 verification.
Before this fix, ``_validate_zip_hash`` returned silently whenever the
``MESH_UPDATE_SHA256`` env var was unset (the default nothing in the
install docs ever told operators to set it). That made the auto-updater
a supply-chain RCE on any compromise of the GitHub release pipeline.
The fix introduces a four-source verification chain:
1. ``MESH_UPDATE_SHA256`` env var (operator override, preserved)
2. ``SHA256SUMS.txt`` asset published alongside the release (primary)
3. Baked-in ``backend/data/release_digests.json`` (fallback)
4. HTTPS-only fallback with a loud warning (preserves auto-update during
transient outages so the user isn't stuck)
A mismatch from any source that DID respond is fatal. Only the "no
source reachable at all" case falls back to HTTPS-only.
"""
import hashlib
import json
from pathlib import Path
import pytest
from services import updater
from services.updater import (
_compute_sha256,
_fetch_sha256sums,
_load_baked_in_release_digests,
_validate_zip_hash,
)
@pytest.fixture
def fake_archive(tmp_path):
"""A tiny synthetic zip-shaped file so we can compute a known digest."""
archive = tmp_path / "update.zip"
payload = b"this is not really a release archive"
archive.write_bytes(payload)
expected = hashlib.sha256(payload).hexdigest().lower()
return str(archive), expected
def test_baked_in_release_digests_file_loads():
"""The shipped release_digests.json must parse and contain v0.9.79."""
digests = _load_baked_in_release_digests()
assert "v0.9.79" in digests
entry = digests["v0.9.79"]
assert "ShadowBroker_v0.9.79.zip" in entry
digest = entry["ShadowBroker_v0.9.79.zip"]
assert len(digest) == 64
assert all(c in "0123456789abcdef" for c in digest)
def test_baked_in_skips_comment_keys():
"""The _comment top-level key is ignored, not surfaced as a release."""
digests = _load_baked_in_release_digests()
assert "_comment" not in digests
def test_compute_sha256_matches_known_value(fake_archive):
archive, expected = fake_archive
assert _compute_sha256(archive) == expected
# ──────────────────────────────────────────────────────────────────────────
# Source 1: MESH_UPDATE_SHA256 env override
# ──────────────────────────────────────────────────────────────────────────
def test_env_override_matching_passes(fake_archive, monkeypatch):
"""Path 1: operator pinned the exact digest via env. Match = success."""
archive, expected = fake_archive
monkeypatch.setenv("MESH_UPDATE_SHA256", expected)
note = _validate_zip_hash(archive)
assert "MESH_UPDATE_SHA256" in note
def test_env_override_mismatch_fails_loudly(fake_archive, monkeypatch):
"""Path 1: operator pinned a different digest. Mismatch = fatal."""
archive, _expected = fake_archive
monkeypatch.setenv("MESH_UPDATE_SHA256", "0" * 64)
with pytest.raises(RuntimeError) as exc_info:
_validate_zip_hash(archive)
assert "mismatch" in str(exc_info.value).lower()
# ──────────────────────────────────────────────────────────────────────────
# Source 2: SHA256SUMS.txt asset
# ──────────────────────────────────────────────────────────────────────────
def test_sha256sums_matching_passes(fake_archive, monkeypatch):
"""Path 2: SHA256SUMS.txt has the correct digest for our asset."""
archive, expected = fake_archive
monkeypatch.delenv("MESH_UPDATE_SHA256", raising=False)
def fake_sums(url):
return {"ShadowBroker_v9.9.9.zip": expected}
monkeypatch.setattr(updater, "_fetch_sha256sums", fake_sums)
note = _validate_zip_hash(
archive,
asset_name="ShadowBroker_v9.9.9.zip",
sha256sums_url="https://example.test/SHA256SUMS.txt",
release_tag="v9.9.9",
)
assert "SHA256SUMS.txt" in note
def test_sha256sums_mismatch_fails_loudly(fake_archive, monkeypatch):
"""Path 2: SHA256SUMS.txt has a different digest. Refuse."""
archive, _expected = fake_archive
monkeypatch.delenv("MESH_UPDATE_SHA256", raising=False)
def fake_sums(url):
return {"ShadowBroker_v9.9.9.zip": "0" * 64}
monkeypatch.setattr(updater, "_fetch_sha256sums", fake_sums)
with pytest.raises(RuntimeError) as exc_info:
_validate_zip_hash(
archive,
asset_name="ShadowBroker_v9.9.9.zip",
sha256sums_url="https://example.test/SHA256SUMS.txt",
release_tag="v9.9.9",
)
assert "mismatch" in str(exc_info.value).lower()
assert "SHA256SUMS" in str(exc_info.value)
# ──────────────────────────────────────────────────────────────────────────
# Source 3: baked-in digest list
# ──────────────────────────────────────────────────────────────────────────
def test_baked_in_matching_passes(fake_archive, monkeypatch):
"""Path 3: SHA256SUMS unreachable, but the baked-in list has us."""
archive, expected = fake_archive
monkeypatch.delenv("MESH_UPDATE_SHA256", raising=False)
monkeypatch.setattr(updater, "_fetch_sha256sums", lambda url: {})
monkeypatch.setattr(
updater,
"_load_baked_in_release_digests",
lambda: {"v9.9.9": {"ShadowBroker_v9.9.9.zip": expected}},
)
note = _validate_zip_hash(
archive,
asset_name="ShadowBroker_v9.9.9.zip",
sha256sums_url="https://example.test/SHA256SUMS.txt",
release_tag="v9.9.9",
)
assert "baked-in" in note
def test_baked_in_mismatch_fails_loudly(fake_archive, monkeypatch):
"""Path 3: baked-in says something different. Refuse."""
archive, _expected = fake_archive
monkeypatch.delenv("MESH_UPDATE_SHA256", raising=False)
monkeypatch.setattr(updater, "_fetch_sha256sums", lambda url: {})
monkeypatch.setattr(
updater,
"_load_baked_in_release_digests",
lambda: {"v9.9.9": {"ShadowBroker_v9.9.9.zip": "0" * 64}},
)
with pytest.raises(RuntimeError) as exc_info:
_validate_zip_hash(
archive,
asset_name="ShadowBroker_v9.9.9.zip",
sha256sums_url="",
release_tag="v9.9.9",
)
assert "mismatch" in str(exc_info.value).lower()
# ──────────────────────────────────────────────────────────────────────────
# Source 4: HTTPS-only fallback
# ──────────────────────────────────────────────────────────────────────────
def test_https_only_fallback_when_no_source_available(fake_archive, monkeypatch, caplog):
"""Path 4: nothing matches — fall back to HTTPS-only with loud warning.
This preserves the auto-update flow during transient outages: an
operator on a flaky network during update doesn't get a hostile
error, they get a degraded-but-functional update with a clear log
message.
"""
import logging
archive, _expected = fake_archive
monkeypatch.delenv("MESH_UPDATE_SHA256", raising=False)
monkeypatch.setattr(updater, "_fetch_sha256sums", lambda url: {})
monkeypatch.setattr(updater, "_load_baked_in_release_digests", lambda: {})
with caplog.at_level(logging.WARNING):
note = _validate_zip_hash(
archive,
asset_name="ShadowBroker_v99.99.zip",
sha256sums_url="",
release_tag="v99.99",
)
assert "https-only" in note.lower()
assert any(
"fell back to HTTPS-only" in rec.getMessage() for rec in caplog.records
)
def test_https_only_fallback_when_release_tag_unknown(fake_archive, monkeypatch):
"""Path 4 also kicks in when we have a baked-in list but it doesn't
contain THIS release tag e.g. a brand-new release that the local
install hasn't seen a digest for yet."""
archive, _expected = fake_archive
monkeypatch.delenv("MESH_UPDATE_SHA256", raising=False)
monkeypatch.setattr(updater, "_fetch_sha256sums", lambda url: {})
monkeypatch.setattr(
updater,
"_load_baked_in_release_digests",
lambda: {"v0.0.1": {"old.zip": "0" * 64}}, # different tag, doesn't match
)
note = _validate_zip_hash(
archive,
asset_name="ShadowBroker_v99.99.zip",
sha256sums_url="",
release_tag="v99.99",
)
assert "https-only" in note.lower()
# ──────────────────────────────────────────────────────────────────────────
# Precedence (env > SHA256SUMS > baked-in > https-only)
# ──────────────────────────────────────────────────────────────────────────
def test_env_override_beats_all_other_sources(fake_archive, monkeypatch):
"""When MESH_UPDATE_SHA256 is set, it's the only source consulted.
The other sources may return false positives or negatives they
shouldn't be queried at all when the operator pinned an exact value.
"""
archive, expected = fake_archive
monkeypatch.setenv("MESH_UPDATE_SHA256", expected)
def boom_sums(url):
raise AssertionError("SHA256SUMS source was queried despite env override")
def boom_baked():
raise AssertionError("Baked-in list was queried despite env override")
monkeypatch.setattr(updater, "_fetch_sha256sums", boom_sums)
monkeypatch.setattr(updater, "_load_baked_in_release_digests", boom_baked)
note = _validate_zip_hash(
archive,
asset_name="any.zip",
sha256sums_url="https://example.test/SHA256SUMS.txt",
release_tag="any",
)
assert "MESH_UPDATE_SHA256" in note
# ──────────────────────────────────────────────────────────────────────────
# _fetch_sha256sums parser
# ──────────────────────────────────────────────────────────────────────────
def test_fetch_sha256sums_parses_standard_format(monkeypatch):
"""Standard ``sha256sum`` output: ``<digest> <filename>``."""
class _Resp:
text = (
"f6877c1d66614525315ea82636ce9f7b41178332c4dbf90d27431a1ea1d9cd47 ShadowBroker_v0.9.79.zip\n"
"e0713c3cdda184cfbea750bfac0d62a35678fec00847e6476f2cac8e7e42046e ShadowBroker_0.9.79_x64_en-US.msi\n"
)
def raise_for_status(self):
pass
def fake_get(url, timeout=15):
return _Resp()
monkeypatch.setattr(updater.requests, "get", fake_get)
monkeypatch.setattr(updater, "_validate_update_url", lambda url, **kw: url)
sums = _fetch_sha256sums("https://example.test/SHA256SUMS.txt")
assert sums["ShadowBroker_v0.9.79.zip"].startswith("f6877c1d")
assert sums["ShadowBroker_0.9.79_x64_en-US.msi"].startswith("e0713c3c")
def test_fetch_sha256sums_handles_binary_marker(monkeypatch):
"""sha256sum -b output: ``<digest> *<filename>``."""
class _Resp:
text = "f6877c1d66614525315ea82636ce9f7b41178332c4dbf90d27431a1ea1d9cd47 *ShadowBroker_v0.9.79.zip\n"
def raise_for_status(self):
pass
monkeypatch.setattr(updater.requests, "get", lambda url, timeout=15: _Resp())
monkeypatch.setattr(updater, "_validate_update_url", lambda url, **kw: url)
sums = _fetch_sha256sums("https://example.test/SHA256SUMS.txt")
assert "ShadowBroker_v0.9.79.zip" in sums
def test_fetch_sha256sums_skips_malformed_lines(monkeypatch):
"""Lines that don't parse cleanly are ignored, not aborted on."""
class _Resp:
text = (
"# comment line\n"
"\n"
"not-a-digest bogus.txt\n"
"f6877c1d66614525315ea82636ce9f7b41178332c4dbf90d27431a1ea1d9cd47 good.zip\n"
)
def raise_for_status(self):
pass
monkeypatch.setattr(updater.requests, "get", lambda url, timeout=15: _Resp())
monkeypatch.setattr(updater, "_validate_update_url", lambda url, **kw: url)
sums = _fetch_sha256sums("https://example.test/SHA256SUMS.txt")
assert "good.zip" in sums
assert "bogus.txt" not in sums
def test_fetch_sha256sums_handles_network_failure(monkeypatch):
"""If the SHA256SUMS asset can't be fetched, return empty (caller
falls through to baked-in / https-only)."""
import requests as _req
def fake_get(url, timeout=15):
raise _req.exceptions.ConnectionError("upstream down")
monkeypatch.setattr(updater.requests, "get", fake_get)
monkeypatch.setattr(updater, "_validate_update_url", lambda url, **kw: url)
sums = _fetch_sha256sums("https://example.test/SHA256SUMS.txt")
assert sums == {}
+14
View File
@@ -28,6 +28,15 @@ services:
- MESH_RELAY_PEERS=${MESH_RELAY_PEERS:-}
# Shared transport auth for operator peer push. Must be set to a unique secret per deployment.
- MESH_PEER_PUSH_SECRET=${MESH_PEER_PUSH_SECRET:-}
# Issue #256: optional per-peer HMAC secrets. Comma-separated
# `url=secret` pairs (no spaces). When a peer URL appears here, only
# the listed per-peer secret is accepted for it — the global
# MESH_PEER_PUSH_SECRET above is ignored for that specific URL. This
# closes the cross-peer impersonation surface for multi-peer fleets.
# Single-peer installs leave this empty (default) for unchanged
# behavior. Both sides of a peering must agree on the per-peer
# secret for a given URL.
- MESH_PEER_SECRETS=${MESH_PEER_SECRETS:-}
# Meshtastic MQTT is opt-in to avoid passive load on the public broker.
# Set MESH_MQTT_ENABLED=true in .env only when this node should join live MQTT.
- MESH_MQTT_ENABLED=${MESH_MQTT_ENABLED:-false}
@@ -43,6 +52,11 @@ services:
# The bundled Docker UI talks to the backend across Docker's private bridge.
# Treat that bridge as local operator access while ports remain bound to 127.0.0.1 by default.
- SHADOWBROKER_TRUST_DOCKER_BRIDGE_LOCAL_OPERATOR=${SHADOWBROKER_TRUST_DOCKER_BRIDGE_LOCAL_OPERATOR:-1}
# Issue #250: bridge trust is now bound to specific container hostnames
# (default: 'frontend' compose service + 'shadowbroker-frontend' container
# name). If you rename the frontend service or run with a different
# container_name, list the hostnames here (comma-separated, no spaces).
- SHADOWBROKER_TRUSTED_FRONTEND_HOSTS=${SHADOWBROKER_TRUSTED_FRONTEND_HOSTS:-frontend,shadowbroker-frontend}
volumes:
- backend_data:/app/data
restart: unless-stopped
@@ -0,0 +1,164 @@
/**
* Issues #218 / #219 / #220 (tg12 external audit):
*
* Every browser-direct call to Wikipedia or Wikidata must send the
* `Api-User-Agent` header that Wikimedia's UA policy asks for. These
* tests pin that requirement on the shared `lib/wikimediaClient`
* helper that WikiImage, NewsFeed, and useRegionDossier all route
* through, so a future refactor that drops the header gets a loud
* test failure rather than a silent ToS regression.
*/
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import {
WIKIMEDIA_API_USER_AGENT,
fetchWikipediaSummary,
fetchWikidataSparql,
_resetWikimediaClientCacheForTests,
} from '@/lib/wikimediaClient';
const originalFetch = globalThis.fetch;
describe('lib/wikimediaClient', () => {
beforeEach(() => {
_resetWikimediaClientCacheForTests();
});
afterEach(() => {
globalThis.fetch = originalFetch;
vi.restoreAllMocks();
});
it('exposes a stable Api-User-Agent identifier with a contact path', () => {
expect(WIKIMEDIA_API_USER_AGENT).toContain('Shadowbroker');
expect(WIKIMEDIA_API_USER_AGENT.toLowerCase()).toContain('github.com');
expect(WIKIMEDIA_API_USER_AGENT.toLowerCase()).toContain('issues');
});
it('sends Api-User-Agent on Wikipedia summary fetch', async () => {
const calls: Array<{ url: string; init?: RequestInit }> = [];
globalThis.fetch = vi.fn(async (url: any, init?: RequestInit) => {
calls.push({ url: String(url), init });
return new Response(
JSON.stringify({
type: 'standard',
title: 'Boeing 747',
description: 'aircraft',
extract: 'long extract',
thumbnail: { source: 'https://example.org/thumb.jpg' },
}),
{ status: 200 },
);
}) as any;
const summary = await fetchWikipediaSummary('Boeing 747');
expect(summary?.thumbnail).toBe('https://example.org/thumb.jpg');
expect(calls).toHaveLength(1);
const headers = (calls[0].init?.headers || {}) as Record<string, string>;
expect(headers['Api-User-Agent']).toBe(WIKIMEDIA_API_USER_AGENT);
});
it('sends Api-User-Agent on Wikidata SPARQL fetch', async () => {
const calls: Array<{ url: string; init?: RequestInit }> = [];
globalThis.fetch = vi.fn(async (url: any, init?: RequestInit) => {
calls.push({ url: String(url), init });
return new Response(
JSON.stringify({
results: {
bindings: [
{
leaderLabel: { value: 'Test Leader' },
govTypeLabel: { value: 'Test Government' },
},
],
},
}),
{ status: 200 },
);
}) as any;
const bindings = await fetchWikidataSparql('SELECT * WHERE { ?s ?p ?o }');
expect(bindings).toHaveLength(1);
const headers = (calls[0].init?.headers || {}) as Record<string, string>;
expect(headers['Api-User-Agent']).toBe(WIKIMEDIA_API_USER_AGENT);
expect(headers['Accept']).toBe('application/sparql-results+json');
});
it('shares cache across consecutive callers for the same Wikipedia title', async () => {
let fetchCount = 0;
globalThis.fetch = vi.fn(async () => {
fetchCount++;
return new Response(
JSON.stringify({
type: 'standard',
title: 'Eiffel Tower',
description: 'iron lattice tower',
extract: '...',
thumbnail: { source: 'https://example.org/eiffel.jpg' },
}),
{ status: 200 },
);
}) as any;
const a = await fetchWikipediaSummary('Eiffel Tower');
const b = await fetchWikipediaSummary('Eiffel Tower');
expect(fetchCount).toBe(1);
expect(a?.thumbnail).toBe(b?.thumbnail);
});
it('deduplicates concurrent in-flight requests for the same title', async () => {
let fetchCount = 0;
globalThis.fetch = vi.fn(async () => {
fetchCount++;
await new Promise((r) => setTimeout(r, 5));
return new Response(
JSON.stringify({
type: 'standard',
title: 'Mount Fuji',
description: 'stratovolcano',
extract: '...',
thumbnail: { source: 'https://example.org/fuji.jpg' },
}),
{ status: 200 },
);
}) as any;
const [a, b, c] = await Promise.all([
fetchWikipediaSummary('Mount Fuji'),
fetchWikipediaSummary('Mount Fuji'),
fetchWikipediaSummary('Mount Fuji'),
]);
expect(fetchCount).toBe(1);
expect(a?.thumbnail).toBe('https://example.org/fuji.jpg');
expect(b).toEqual(a);
expect(c).toEqual(a);
});
it('returns null on disambiguation pages without throwing', async () => {
globalThis.fetch = vi.fn(async () =>
new Response(JSON.stringify({ type: 'disambiguation' }), { status: 200 }),
) as any;
const summary = await fetchWikipediaSummary('Mercury');
expect(summary).toBeNull();
});
it('returns null on HTTP error without throwing', async () => {
globalThis.fetch = vi.fn(async () => new Response('not found', { status: 404 })) as any;
const summary = await fetchWikipediaSummary('Nonexistent Article 12345');
expect(summary).toBeNull();
});
it('returns null on network error without throwing', async () => {
globalThis.fetch = vi.fn(async () => {
throw new Error('network down');
}) as any;
const summary = await fetchWikipediaSummary('Anything');
expect(summary).toBeNull();
});
it('returns null on empty input', async () => {
globalThis.fetch = vi.fn(async () => new Response('{}', { status: 200 })) as any;
expect(await fetchWikipediaSummary('')).toBeNull();
expect(await fetchWikipediaSummary(' ')).toBeNull();
expect(globalThis.fetch).not.toHaveBeenCalled();
});
});
+24 -20
View File
@@ -5,6 +5,7 @@ import { motion, AnimatePresence } from 'framer-motion';
import { AlertTriangle, Clock, Minus, Plus, ExternalLink, Brain, Loader2 } from 'lucide-react';
import React, { useEffect, useRef, useCallback } from 'react';
import WikiImage from '@/components/WikiImage';
import { fetchWikipediaSummary } from '@/lib/wikimediaClient';
import type { SelectedEntity, RegionDossier, FimiData } from "@/types/dashboard";
import { useDataKeys } from '@/hooks/useDataStore';
import { API_BASE } from '@/lib/api';
@@ -203,34 +204,37 @@ function resolveAircraftWikiTitle(model: string | undefined): string | null {
return AIRCRAFT_WIKI[model] || resolveAcTypeWiki(model);
}
// Module-level cache for Wikipedia thumbnails (persists across re-renders)
const _wikiThumbCache: Record<string, { url: string | null; loading: boolean }> = {};
// Issue #220 (tg12): the previous implementation kept its own
// module-local Wikipedia thumbnail cache and issued anonymous fetches
// without `Api-User-Agent`. We now delegate to lib/wikimediaClient,
// which sends the policy-compliant header and shares one cache with
// WikiImage and useRegionDossier.
function useAircraftImage(model: string | undefined): { imgUrl: string | null; wikiUrl: string | null; loading: boolean } {
const [, forceUpdate] = useState(0);
const [imgUrl, setImgUrl] = useState<string | null>(null);
const [loading, setLoading] = useState(false);
const wikiTitle = resolveAircraftWikiTitle(model) || undefined;
const wikiUrl = wikiTitle ? `https://en.wikipedia.org/wiki/${wikiTitle.replace(/ /g, '_')}` : null;
useEffect(() => {
if (!wikiTitle) return;
const key = wikiTitle;
if (_wikiThumbCache[key]) return; // Already fetched or in-flight
_wikiThumbCache[key] = { url: null, loading: true };
fetch(`https://en.wikipedia.org/api/rest_v1/page/summary/${encodeURIComponent(wikiTitle)}`)
.then(r => r.json())
.then(d => {
_wikiThumbCache[key] = { url: d.thumbnail?.source || null, loading: false };
forceUpdate(n => n + 1);
})
.catch(() => {
_wikiThumbCache[key] = { url: null, loading: false };
forceUpdate(n => n + 1);
});
let cancelled = false;
if (!wikiTitle) {
setImgUrl(null);
setLoading(false);
return;
}
setLoading(true);
fetchWikipediaSummary(wikiTitle).then((summary) => {
if (cancelled) return;
setImgUrl(summary?.thumbnail || null);
setLoading(false);
});
return () => {
cancelled = true;
};
}, [wikiTitle]);
if (!wikiTitle) return { imgUrl: null, wikiUrl: null, loading: false };
const cached = _wikiThumbCache[wikiTitle];
return { imgUrl: cached?.url || null, wikiUrl, loading: cached?.loading || false };
return { imgUrl, wikiUrl, loading };
}
+25 -23
View File
@@ -1,13 +1,17 @@
'use client';
import React, { useState, useEffect } from 'react';
import ExternalImage from '@/components/ExternalImage';
// Module-level cache: Wikipedia article title → thumbnail URL
const _cache: Record<string, { url: string | null; done: boolean }> = {};
import { fetchWikipediaSummary } from '@/lib/wikimediaClient';
/**
* WikiImage displays a Wikipedia thumbnail for a given article URL.
* Uses the Wikipedia REST API with a module-level cache (only fetches once per article).
*
* Issue #220 (tg12): this component previously had its own
* module-local Wikipedia fetch + cache. It now delegates to
* `lib/wikimediaClient`, which sends the policy-compliant
* `Api-User-Agent` header and shares one cache across every UI
* component that asks Wikipedia for an article summary (WikiImage,
* NewsFeed, useRegionDossier).
*
* Props:
* wikiUrl: Full Wikipedia URL, e.g. "https://en.wikipedia.org/wiki/Boeing_787_Dreamliner"
@@ -26,32 +30,30 @@ export default function WikiImage({
maxH?: string;
accent?: string;
}) {
const [, forceUpdate] = useState(0);
const [imgUrl, setImgUrl] = useState<string | null>(null);
const [loading, setLoading] = useState(true);
// Extract article title from URL
const title = wikiUrl.replace(/^https?:\/\/[^/]+\/wiki\//, '');
useEffect(() => {
if (!title || _cache[title]?.done) return;
if (_cache[title]) return; // In-flight
_cache[title] = { url: null, done: false };
fetch(`https://en.wikipedia.org/api/rest_v1/page/summary/${encodeURIComponent(title)}`)
.then((r) => r.json())
.then((d) => {
_cache[title] = { url: d.thumbnail?.source || d.originalimage?.source || null, done: true };
forceUpdate((n) => n + 1);
})
.catch(() => {
_cache[title] = { url: null, done: true };
forceUpdate((n) => n + 1);
});
let cancelled = false;
if (!title) {
setImgUrl(null);
setLoading(false);
return;
}
setLoading(true);
fetchWikipediaSummary(title).then((summary) => {
if (cancelled) return;
setImgUrl(summary?.thumbnail || null);
setLoading(false);
});
return () => {
cancelled = true;
};
}, [title]);
const cached = _cache[title];
const imgUrl = cached?.url;
const loading = cached && !cached.done;
return (
<div className="pb-2">
{loading && (
+23 -22
View File
@@ -1,5 +1,6 @@
import { useCallback, useState, useEffect } from 'react';
import type { RegionDossier, SelectedEntity } from '@/types/dashboard';
import { fetchWikipediaSummary, fetchWikidataSparql } from '@/lib/wikimediaClient';
// ─── CACHE ─────────────────────────────────────────────────────────────────
// Simple in-memory cache keyed by rounded lat/lng (0.1° ≈ 11km grid), 24h TTL.
@@ -114,7 +115,11 @@ async function fetchCountryData(countryCode: string) {
return Array.isArray(data) ? data[0] || {} : data || {};
}
/** Fetch head of state + government type from Wikidata SPARQL (direct browser call). */
/** Fetch head of state + government type from Wikidata SPARQL.
*
* Issue #218 (tg12): routes through lib/wikimediaClient so the
* Api-User-Agent header is set per Wikimedia's UA policy.
*/
async function fetchLeader(countryName: string) {
if (!countryName) return { leader: 'Unknown', government_type: 'Unknown' };
const safeName = countryName.replace(/"/g, '\\"').replace(/'/g, "\\'");
@@ -127,13 +132,11 @@ async function fetchLeader(countryName: string) {
SERVICE wikibase:label { bd:serviceParam wikibase:language "en". }
} LIMIT 1
`;
const url = `https://query.wikidata.org/sparql?query=${encodeURIComponent(sparql)}&format=json`;
const res = await fetch(url, {
headers: { Accept: 'application/sparql-results+json' },
});
if (!res.ok) throw new Error(`Wikidata HTTP ${res.status}`);
const results = (await res.json()).results?.bindings || [];
if (results.length > 0) {
const results = await fetchWikidataSparql<{
leaderLabel?: { value: string };
govTypeLabel?: { value: string };
}>(sparql);
if (results && results.length > 0) {
return {
leader: results[0].leaderLabel?.value || 'Unknown',
government_type: results[0].govTypeLabel?.value || 'Unknown',
@@ -142,27 +145,25 @@ async function fetchLeader(countryName: string) {
return { leader: 'Unknown', government_type: 'Unknown' };
}
/** Fetch Wikipedia summary for a place (direct browser call). */
/** Fetch Wikipedia summary for a place.
*
* Issue #219 (tg12): routes through lib/wikimediaClient so the
* Api-User-Agent header is set per Wikimedia's UA policy, AND the
* shared cache means consecutive useRegionDossier + WikiImage +
* NewsFeed lookups for the same article all hit the same slot.
*/
async function fetchLocalWikiSummary(placeName: string, countryName = '') {
if (!placeName) return {};
const candidates = [placeName];
if (countryName) candidates.push(`${placeName}, ${countryName}`);
for (const name of candidates) {
try {
const slug = encodeURIComponent(name.replace(/ /g, '_'));
const url = `https://en.wikipedia.org/api/rest_v1/page/summary/${slug}`;
const res = await fetch(url);
if (!res.ok) continue;
const data = await res.json();
if (data.type === 'disambiguation') continue;
const summary = await fetchWikipediaSummary(name);
if (summary) {
return {
description: data.description || '',
extract: data.extract || '',
thumbnail: data.thumbnail?.source || '',
description: summary.description,
extract: summary.extract,
thumbnail: summary.thumbnail,
};
} catch {
continue;
}
}
return {};
+157
View File
@@ -0,0 +1,157 @@
/**
* wikimediaClient single fetch surface for Wikipedia / Wikidata.
*
* Issues #218, #219, #220 (tg12 external audit):
*
* Wikimedia's User-Agent policy asks API clients to identify themselves
* via `Api-User-Agent` when calling from browser JavaScript (because the
* browser does not let JS set `User-Agent` directly). Before this
* module existed, three independent components issued anonymous browser
* fetches against Wikipedia / Wikidata:
*
* - useRegionDossier (Wikidata SPARQL + Wikipedia REST summary)
* - WikiImage (Wikipedia REST summary)
* - NewsFeed (Wikipedia REST summary)
*
* Each component shipped its own copy-pasted fetch + module-local cache.
* Provider-policy compliance was missing in all three places.
*
* This module centralizes:
*
* 1. The `Api-User-Agent` header on every request.
* 2. A single LRU cache for Wikipedia summary lookups (keyed by article
* title). Multiple components asking for the same article share
* one in-flight request and one cache slot.
* 3. One predictable kill switch if Wikimedia ever asks us to back
* off, we change `WIKIMEDIA_API_USER_AGENT` here and the whole
* frontend updates.
*
* This does NOT change end-user UX:
*
* - WikiImage still shows the same thumbnails.
* - NewsFeed still shows aircraft thumbnails.
* - useRegionDossier still returns the same place summary + leader.
*
* What changes:
*
* - Wikimedia can identify our traffic from any other anonymous
* browser visitor pool.
* - Provider-policy fixes happen here once, not in three places.
*/
// Stable identifier per Wikimedia UA policy. Includes a contact path so
// Wikimedia's operators can reach the project if they need to rate-limit
// or coordinate. Bump the version when the contact path changes.
export const WIKIMEDIA_API_USER_AGENT =
'Shadowbroker/1.0 (+https://github.com/BigBodyCobain/Shadowbroker; ' +
'report issues at /issues)';
// Module-level cache shared by WikiImage, NewsFeed, and useRegionDossier.
// Keyed by Wikipedia article title (NOT slug — we keep the human-readable
// form so debugging the cache is easier). Values track in-flight state
// so concurrent callers for the same title share one network request.
export interface WikipediaSummary {
title: string;
description: string;
extract: string;
thumbnail: string;
type: string; // 'standard' | 'disambiguation' | etc.
}
interface CacheEntry {
summary: WikipediaSummary | null;
inflight: Promise<WikipediaSummary | null> | null;
loaded: boolean;
}
const _summaryCache: Map<string, CacheEntry> = new Map();
const SUMMARY_CACHE_MAX = 512;
function evictIfOverCap() {
if (_summaryCache.size <= SUMMARY_CACHE_MAX) return;
const oldest = _summaryCache.keys().next().value;
if (oldest) _summaryCache.delete(oldest);
}
/** Fetch a Wikipedia article summary (titles, NOT URLs).
*
* Empty / invalid input resolves to `null`. Network errors and disambig
* pages also resolve to `null` so callers can render a fallback without
* a try/catch. Per the audit's "fail forward, not loud" rule.
*/
export async function fetchWikipediaSummary(
title: string,
): Promise<WikipediaSummary | null> {
const trimmed = (title || '').trim();
if (!trimmed) return null;
const cached = _summaryCache.get(trimmed);
if (cached?.loaded) return cached.summary;
if (cached?.inflight) return cached.inflight;
const slug = encodeURIComponent(trimmed.replace(/ /g, '_'));
const url = `https://en.wikipedia.org/api/rest_v1/page/summary/${slug}`;
const promise = fetch(url, {
headers: { 'Api-User-Agent': WIKIMEDIA_API_USER_AGENT },
})
.then(async (r) => {
if (!r.ok) return null;
const d = await r.json();
if (d?.type === 'disambiguation') return null;
const summary: WikipediaSummary = {
title: trimmed,
description: d?.description || '',
extract: d?.extract || '',
thumbnail: d?.thumbnail?.source || d?.originalimage?.source || '',
type: d?.type || 'standard',
};
return summary;
})
.catch(() => null)
.then((summary) => {
_summaryCache.set(trimmed, { summary, inflight: null, loaded: true });
evictIfOverCap();
return summary;
});
_summaryCache.set(trimmed, { summary: null, inflight: promise, loaded: false });
evictIfOverCap();
return promise;
}
/** Fetch a Wikidata SPARQL query result.
*
* Returns the parsed JSON `results.bindings` array on success; `null`
* (not throwing) on any failure so callers can render fallbacks
* silently. Kept as a thin wrapper so the audit-required UA header is
* applied in exactly one place.
*/
export async function fetchWikidataSparql<T = Record<string, { value: string }>>(
sparql: string,
): Promise<T[] | null> {
const trimmed = (sparql || '').trim();
if (!trimmed) return null;
const url = `https://query.wikidata.org/sparql?query=${encodeURIComponent(
trimmed,
)}&format=json`;
try {
const res = await fetch(url, {
headers: {
'Api-User-Agent': WIKIMEDIA_API_USER_AGENT,
Accept: 'application/sparql-results+json',
},
});
if (!res.ok) return null;
const json = await res.json();
const bindings = json?.results?.bindings;
return Array.isArray(bindings) ? (bindings as T[]) : null;
} catch {
return null;
}
}
/** Internal: clear the shared cache. Exposed for tests only. */
export function _resetWikimediaClientCacheForTests() {
_summaryCache.clear();
}
+29
View File
@@ -6,6 +6,35 @@ module.exports = defineConfig({
environment: 'jsdom',
globals: true,
include: ['src/**/*.test.{ts,tsx}'],
// Default test timeout: 15s (up from vitest's 5s default).
//
// We render real React component trees under jsdom in many tests, and
// GitHub Actions' shared Node.js workers (specifically the
// "CI - Lint & Test / Frontend Tests & Build" job) consistently
// measure 610s for the heavier MessagesView / GateView / Wormhole
// contact flows under CI load. On a developer laptop those same tests
// settle in <1s, so the 5s default was tuned to local dev speed and
// not to CI runner speed.
//
// Concrete history that drove this bump (none of these were real
// product bugs — all were CI load racing the 5s ceiling on
// findByText / waitFor against React reconciliation):
// PR #226, #237, #261, #262, #265 all flaked on
// src/__tests__/mesh/messagesViewFirstContact.test.tsx
// src/__tests__/mesh/gateCompatDecryptUx.test.tsx
// PR #262's flake was the worst — it fired on the post-merge
// Docker Publish run and prevented the AIS SPKI security fix's
// image from being published to GHCR until the next PR
// cumulatively re-published it.
//
// 15s is generous enough to absorb routine CI slowness without
// masking real "test never settles" bugs (those would still time
// out, just three rounds later). Individual tests can still pin
// their own tighter timeout via the third arg to `it()`.
testTimeout: 15000,
// Hook timeout follows test timeout — beforeEach/afterEach setup
// for the heavier component tests has the same CI-load sensitivity.
hookTimeout: 15000,
},
resolve: {
alias: {
Generated
+11
View File
@@ -82,6 +82,7 @@ dependencies = [
{ name = "cachetools" },
{ name = "cloudscraper" },
{ name = "cryptography" },
{ name = "defusedxml" },
{ name = "fastapi" },
{ name = "feedparser" },
{ name = "httpx" },
@@ -120,6 +121,7 @@ requires-dist = [
{ name = "cachetools", specifier = "==5.5.2" },
{ name = "cloudscraper", specifier = "==1.2.71" },
{ name = "cryptography", specifier = ">=41.0.0" },
{ name = "defusedxml", specifier = ">=0.7.1" },
{ name = "fastapi", specifier = "==0.115.12" },
{ name = "feedparser", specifier = "==6.0.10" },
{ name = "httpx", specifier = "==0.28.1" },
@@ -600,6 +602,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/a4/87/d03a718e7bfdbbebaa4b6a66ba5bb069bc00a84e5ad176d8198cc785cd42/dbus_fast-4.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:f6af190d8306f1bd506740c39701f5c211aa31ac660a3fcb401ebb97d33166c7", size = 1627620, upload-time = "2026-02-01T21:05:46.878Z" },
]
[[package]]
name = "defusedxml"
version = "0.7.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/0f/d5/c66da9b79e5bdb124974bfe172b4daf3c984ebd9c2a06e2b8a4dc7331c72/defusedxml-0.7.1.tar.gz", hash = "sha256:1bb3032db185915b62d7c6209c5a8792be6a32ab2fedacc84e01b52c51aa3e69", size = 75520, upload-time = "2021-03-08T10:59:26.269Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/07/6c/aa3f2f849e01cb6a001cd8554a88d4c77c5c1a31c95bdf1cf9301e6d9ef4/defusedxml-0.7.1-py2.py3-none-any.whl", hash = "sha256:a352e7e428770286cc899e2542b6cdaedb2b4953ff269a210103ec58f6198a61", size = 25604, upload-time = "2021-03-08T10:59:24.45Z" },
]
[[package]]
name = "deprecated"
version = "1.3.1"