CalvinBackup/Shadowbroker
1
0
Fork 0
mirror of https://github.com/BigBodyCobain/Shadowbroker.git synced 2026-06-08 15:23:54 +02:00
Code Issues Packages Projects Releases Wiki Activity

Fix #298: move Sentinel credentials from browser storage to backend .env

Browse Source 
Reported by @tg12. Pre-fix, the Settings panel stored real third-party
Copernicus CDSE client_id + client_secret in browser localStorage /
sessionStorage via the privacy storage helper, and the proxy routes
required those values to come back in every tile/token request body.
Any same-origin script (XSS, malicious browser extension, dev-tools
HAR export) had read access to the credentials.

This change moves them server-side, behind the same .env-backed admin
flow every other third-party API key (OpenSky, AIS Stream, Finnhub,
Shodan, …) already uses.

Backend
-------
backend/services/api_settings.py
  * Added SENTINEL_CLIENT_ID and SENTINEL_CLIENT_SECRET entries to
    API_REGISTRY. The existing GET/PUT /api/settings/api-keys flow
    (already require_local_operator-gated, .env-backed) now manages
    them — no new route surface.

backend/routers/tools.py
  * /api/sentinel/token and /api/sentinel/tile resolve credentials via
    a new _resolve_sentinel_credentials() helper: body fields win for
    back-compat with any legacy callers, otherwise the helper reads
    SENTINEL_CLIENT_ID / SENTINEL_CLIENT_SECRET from os.environ.
  * When neither source has a value, the route returns 400 with a
    friendly pointer ("Set SENTINEL_CLIENT_ID and SENTINEL_CLIENT_SECRET
    in the API Keys panel") instead of the curt "required" message.
    The user's standing rule against hostile errors applies.
  * Function bodies only — decorator lines untouched, so this PR does
    not conflict with #303 (which adds Depends(require_local_operator)
    to the same routes).

Frontend
--------
frontend/src/lib/sentinelHub.ts — rewritten
  * Removed: getSentinelCredentials / setSentinelCredentials /
    clearSentinelCredentials / getSentinelCredentialStorageMode.
    These were the browser-storage read/write helpers; their existence
    was the bug.
  * Added: checkBackendSentinelStatus(), refreshSentinelStatus(),
    getCachedSentinelStatus(), and a kept-for-back-compat
    hasSentinelCredentials() shim. Status is sourced from
    /api/settings/api-keys (the same endpoint the API Keys panel
    already uses), so we don't add a new route just for this read.
  * Added: migrateLegacySentinelBrowserKeys() — one-shot, idempotent
    helper that clears sb_sentinel_client_id / _secret / _instance_id
    from BOTH localStorage and sessionStorage. We deliberately do NOT
    auto-POST those legacy browser values to the backend; doing so
    would silently migrate a secret across a trust boundary without
    operator consent. Operators re-enter once in the API Keys panel
    and the legacy keys get wiped here.
  * fetchSentinelTile and getSentinelToken no longer send client_id /
    client_secret in the request body. The backend uses .env.

frontend/src/components/SettingsPanel.tsx
  * Dropped sb_sentinel_client_id / _secret / _instance_id from
    PRIVACY_SENSITIVE_BROWSER_KEYS — they're no longer written.
  * SentinelTab rewritten: removed the inline Client ID / Client Secret
    inputs + Save / Clear / Test buttons. Replaced with a status panel
    that calls checkBackendSentinelStatus() on mount, a one-click
    "Open API Keys Panel" button, and a migration banner that appears
    only when migrateLegacySentinelBrowserKeys() actually cleared
    something.
  * Setup guide STEP 3 now points to the API Keys panel instead of
    the local form.

frontend/src/app/page.tsx
  * Added a one-time useEffect that fires checkBackendSentinelStatus()
    on mount so the cached value (which the synchronous
    hasSentinelCredentials() shim reads) is populated before
    MaplibreViewer's tile-URL memo runs.

Tests
-----
backend/tests/test_sentinel_credentials_server_side.py (new)
  * API_REGISTRY surface — sentinel_client_id / sentinel_client_secret
    are registered with the right env_keys, ALLOWED_ENV_KEYS lets
    /api/settings/api-keys PUT them.
  * Resolution order — body wins, env is fallback, neither → 400 with
    the friendly pointer message, and NO upstream HTTP call when
    neither source has credentials (asserted via
    MagicMock(side_effect=AssertionError)).
  * /api/sentinel/tile same shape.

frontend/src/__tests__/utils/sentinelHub.test.ts (new)
  * migrateLegacySentinelBrowserKeys clears localStorage AND
    sessionStorage, reports what it cleared, idempotent.
  * fetchSentinelTile + getSentinelToken POST WITHOUT client_id /
    client_secret in the body (plants leaked credentials in browser
    storage first to prove they are NOT picked up).
  * checkBackendSentinelStatus parses /api/settings/api-keys correctly:
    true only when both keys is_set, false on partial config or
    network errors.

All 7 backend tests + 8 frontend tests pass locally. The
test_no_new_duplicate_routes guard and the api-settings test suite
still pass.

Credit: @tg12 for the audit report.
This commit is contained in:
BigBodyCobain
2026-05-22 10:44:50 -06:00
parent 19fb7f0b1e
commit b041b5e97c
7 changed files with 767 additions and 215 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/routers/tools.py
+58 -7
View File
@@ -97,18 +97,57 @@ def api_sentinel2_search(
return search_sentinel2_scene(lat, lng)
# Issue #298 (tg12): Sentinel credentials moved server-side
# ---------------------------------------------------------------------------
# Previously the frontend kept Copernicus CDSE client_id + client_secret in
# browser localStorage / sessionStorage and forwarded them on every tile
# request through this proxy. That exposed real third-party credentials to
# any same-origin script (XSS, malicious browser extension, dev-tools HAR
# export).
#
# Resolution order (first match wins):
# 1. Request body — kept for back-compat. A small number of legacy
# operator setups may still post credentials; we don't break them.
# 2. Backend .env — SENTINEL_CLIENT_ID / SENTINEL_CLIENT_SECRET, managed
# through the existing /api/settings/api-keys flow (admin-gated).
#
# The frontend in ``sentinelHub.ts`` no longer reads browser storage and no
# longer forwards credentials — every dashboard request now lands in (2).
# ---------------------------------------------------------------------------
def _resolve_sentinel_credentials(body_id: str, body_secret: str) -> tuple[str, str]:
"""Return (client_id, client_secret) using body values when present,
otherwise falling back to backend .env. Empty strings if neither is set."""
import os as _os
cid = (body_id or "").strip() or (_os.environ.get("SENTINEL_CLIENT_ID", "") or "").strip()
csec = (body_secret or "").strip() or (_os.environ.get("SENTINEL_CLIENT_SECRET", "") or "").strip()
return cid, csec
@router.post("/api/sentinel/token")
@limiter.limit("60/minute")
async def api_sentinel_token(request: Request):
"""Proxy Copernicus CDSE OAuth2 token request (avoids browser CORS block)."""
"""Proxy Copernicus CDSE OAuth2 token request (avoids browser CORS block).
Credentials are resolved by ``_resolve_sentinel_credentials`` — body
fields are honored for back-compat, otherwise the backend .env values
populated through ``/api/settings/api-keys`` are used.
"""
import requests as req
body = await request.body()
from urllib.parse import parse_qs
params = parse_qs(body.decode("utf-8"))
client_id = params.get("client_id", [""])[0]
client_secret = params.get("client_secret", [""])[0]
body_id = params.get("client_id", [""])[0]
body_secret = params.get("client_secret", [""])[0]
client_id, client_secret = _resolve_sentinel_credentials(body_id, body_secret)
if not client_id or not client_secret:
raise HTTPException(400, "client_id and client_secret required")
# Friendly, non-hostile error — points the operator at the place
# they configure other API keys instead of just saying "required".
raise HTTPException(
400,
"Sentinel client_id/client_secret are not configured. "
"Set SENTINEL_CLIENT_ID and SENTINEL_CLIENT_SECRET in the "
"API Keys panel (Settings → API Keys) or your backend .env.",
)
token_url = "https://identity.dataspace.copernicus.eu/auth/realms/CDSE/protocol/openid-connect/token"
try:
resp = await asyncio.to_thread(req.post, token_url,
@@ -163,8 +202,11 @@ async def api_sentinel_tile(request: Request):
except Exception:
return JSONResponse(status_code=422, content={"ok": False, "detail": "invalid JSON body"})
client_id = body.get("client_id", "")
client_secret = body.get("client_secret", "")
# Issue #298: same resolution order as /api/sentinel/token — body
# values for back-compat, otherwise backend .env.
body_id = body.get("client_id", "")
body_secret = body.get("client_secret", "")
client_id, client_secret = _resolve_sentinel_credentials(body_id, body_secret)
preset = body.get("preset", "TRUE-COLOR")
date_str = body.get("date", "")
z = body.get("z", 0)
@@ -172,7 +214,16 @@ async def api_sentinel_tile(request: Request):
y = body.get("y", 0)
if not client_id or not client_secret or not date_str:
raise HTTPException(400, "client_id, client_secret, and date required")
# Distinguish "no creds" from "no date" so the operator knows
# what to fix. Same friendly pointer as the /token route.
if not client_id or not client_secret:
raise HTTPException(
400,
"Sentinel client_id/client_secret are not configured. "
"Set SENTINEL_CLIENT_ID and SENTINEL_CLIENT_SECRET in the "
"API Keys panel (Settings → API Keys) or your backend .env.",
)
raise HTTPException(400, "date required")
now = _time.time()
credential_fp = _credential_fingerprint(client_id, client_secret)
backend/services/api_settings.py
+25
View File
@@ -150,6 +150,31 @@ API_REGISTRY = [
"url": "https://finnhub.io/register",
"required": False,
},
# Issue #298 (tg12): Sentinel Hub / Copernicus Data Space Ecosystem
# credentials were previously held in browser localStorage / sessionStorage
# by the Settings panel. Moved server-side to the same .env-backed
# store every other third-party API key lives in. The Sentinel proxy
# routes (POST /api/sentinel/token, /tile) now fall back to these
# env values when the request body omits credentials — see
# backend/routers/tools.py for the resolution order.
{
"id": "sentinel_client_id",
"env_key": "SENTINEL_CLIENT_ID",
"name": "Sentinel Hub / Copernicus — Client ID",
"description": "OAuth2 client ID for Copernicus Data Space Ecosystem (CDSE). Required for the Sentinel-2 imagery overlay and the right-click Sentinel-2 Intel Card. Sign in at dataspace.copernicus.eu and create OAuth credentials.",
"category": "Imagery",
"url": "https://dataspace.copernicus.eu/",
"required": False,
},
{
"id": "sentinel_client_secret",
"env_key": "SENTINEL_CLIENT_SECRET",
"name": "Sentinel Hub / Copernicus — Client Secret",
"description": "OAuth2 client secret paired with the Client ID above. Used by the backend to mint short-lived access tokens against the CDSE identity provider. Stored in the backend .env; never sent to the browser.",
"category": "Imagery",
"url": "https://dataspace.copernicus.eu/",
"required": False,
},
]
ALLOWED_ENV_KEYS = {
backend/tests/test_sentinel_credentials_server_side.py
+277
View File
@@ -0,0 +1,277 @@
"""Issue #298 (tg12): Sentinel credentials must live server-side.
Before the fix, ``frontend/src/components/SettingsPanel.tsx`` stored
``client_id`` and ``client_secret`` in ``localStorage`` /
``sessionStorage`` via the privacy storage helper, and the proxy routes
in ``backend/routers/tools.py`` REQUIRED those values to come in the
request body. Any same-origin script (XSS, malicious extension,
dev-tools HAR export) had read access to real third-party Sentinel
credentials.
After the fix:
* ``SENTINEL_CLIENT_ID`` and ``SENTINEL_CLIENT_SECRET`` are entries
in the ``api_settings.API_REGISTRY`` and are persisted via the
existing ``/api/settings/api-keys`` flow (admin-gated, .env-backed,
never returned to the browser).
* The proxy routes prefer request-body values for back-compat but
fall back to ``os.environ.get("SENTINEL_CLIENT_ID")`` /
``os.environ.get("SENTINEL_CLIENT_SECRET")`` when the body omits
them. The dashboard's ``sentinelHub.ts`` no longer sends credentials
in the body — every request now hits the env path.
* When neither source has a value, the route returns a 400 with a
pointer to the API Keys panel rather than a curt "client_id and
client_secret required" message.
These tests cover the resolution order and the registry surface.
"""
from __future__ import annotations
from unittest.mock import patch, MagicMock
import pytest
# ---------------------------------------------------------------------------
# Helper: import the routes module fresh per test so monkey-patched
# environment variables are picked up by the route's os.environ.get call.
# (The lookup is per-request, not at import time, so this isn't strictly
# required — but it makes the test layout obvious.)
# ---------------------------------------------------------------------------
@pytest.fixture
def loopback_client():
"""ASGI client with peer IP 127.0.0.1 so the Sentinel routes' (post-#303)
``require_local_operator`` gate passes.
Built without a context manager so the privacy-core lifespan check
doesn't run in the test env.
"""
import asyncio
from httpx import ASGITransport, AsyncClient
from main import app
class _Loop:
def __init__(self):
self._loop = asyncio.new_event_loop()
self._transport = ASGITransport(app=app, client=("127.0.0.1", 12345))
self._base = "http://127.0.0.1:8000"
def _do(self, method: str, url: str, **kw):
async def go():
async with AsyncClient(transport=self._transport, base_url=self._base) as ac:
return await ac.request(method, url, **kw)
return self._loop.run_until_complete(go())
def get(self, url, **kw): return self._do("GET", url, **kw)
def post(self, url, **kw): return self._do("POST", url, **kw)
def put(self, url, **kw): return self._do("PUT", url, **kw)
def close(self): self._loop.close()
c = _Loop()
yield c
c.close()
# ---------------------------------------------------------------------------
# API_REGISTRY surface
# ---------------------------------------------------------------------------
class TestApiRegistry:
def test_sentinel_keys_registered(self):
"""Both Sentinel keys must be entries in API_REGISTRY so the
existing /api/settings/api-keys PUT flow can write them to .env."""
from services.api_settings import API_REGISTRY, ALLOWED_ENV_KEYS
ids = {row["id"] for row in API_REGISTRY}
assert "sentinel_client_id" in ids
assert "sentinel_client_secret" in ids
# Critical: ALLOWED_ENV_KEYS is the gate on which .env keys the
# API can mutate. If we forgot to add the env_key field on the
# registry rows, callers couldn't actually save the values.
assert "SENTINEL_CLIENT_ID" in ALLOWED_ENV_KEYS
assert "SENTINEL_CLIENT_SECRET" in ALLOWED_ENV_KEYS
def test_api_keys_put_accepts_sentinel_keys(self, loopback_client, monkeypatch, tmp_path):
"""End-to-end: PUT /api/settings/api-keys with SENTINEL_CLIENT_ID
+ SENTINEL_CLIENT_SECRET must persist to .env."""
import services.api_settings as api_settings
# Redirect both .env paths to tmp so the test doesn't mutate
# the developer's real backend .env.
tmp_env = tmp_path / ".env"
monkeypatch.setattr(api_settings, "ENV_PATH", tmp_env)
monkeypatch.setattr(api_settings, "OPERATOR_KEYS_ENV_PATH", tmp_path / "operator_api_keys.env")
r = loopback_client.put(
"/api/settings/api-keys",
json={
"SENTINEL_CLIENT_ID": "test-sentinel-id",
"SENTINEL_CLIENT_SECRET": "test-sentinel-secret",
},
)
assert r.status_code == 200, f"PUT failed: {r.text}"
body = r.json()
assert body.get("ok") is True
# File on disk should now carry both keys.
parsed = api_settings._parse_env_file(tmp_env)
assert parsed.get("SENTINEL_CLIENT_ID") == "test-sentinel-id"
assert parsed.get("SENTINEL_CLIENT_SECRET") == "test-sentinel-secret"
# ---------------------------------------------------------------------------
# Credential resolution — body wins, env is fallback, neither is 400
# ---------------------------------------------------------------------------
class TestSentinelTokenCredResolution:
def test_env_fallback_when_body_empty(self, loopback_client, monkeypatch):
"""No body credentials → backend reads .env values."""
monkeypatch.setenv("SENTINEL_CLIENT_ID", "env-id")
monkeypatch.setenv("SENTINEL_CLIENT_SECRET", "env-secret")
# Mock the upstream Copernicus call so we don't hit the network.
# Capture what was sent so we can prove env values were used.
captured: dict = {}
fake_resp = MagicMock()
fake_resp.status_code = 200
fake_resp.content = b'{"access_token": "stub", "expires_in": 300}'
def fake_post(url, *args, **kwargs):
captured["url"] = url
captured["data"] = kwargs.get("data", {})
return fake_resp
with patch("requests.post", side_effect=fake_post):
r = loopback_client.post(
"/api/sentinel/token",
data={}, # ← deliberately empty body
headers={"Content-Type": "application/x-www-form-urlencoded"},
)
assert r.status_code == 200
# The forwarded creds must come from env, not from a stale cache
# or fallback string.
assert captured.get("data", {}).get("client_id") == "env-id"
assert captured.get("data", {}).get("client_secret") == "env-secret"
def test_body_credentials_win_over_env(self, loopback_client, monkeypatch):
"""Body values (back-compat path) must win when both sources
are present. This preserves the pre-#298 behavior for any
legacy callers that still post credentials."""
monkeypatch.setenv("SENTINEL_CLIENT_ID", "env-id")
monkeypatch.setenv("SENTINEL_CLIENT_SECRET", "env-secret")
captured: dict = {}
fake_resp = MagicMock()
fake_resp.status_code = 200
fake_resp.content = b'{"access_token": "stub"}'
def fake_post(url, *args, **kwargs):
captured["data"] = kwargs.get("data", {})
return fake_resp
with patch("requests.post", side_effect=fake_post):
r = loopback_client.post(
"/api/sentinel/token",
data={"client_id": "body-id", "client_secret": "body-secret"},
headers={"Content-Type": "application/x-www-form-urlencoded"},
)
assert r.status_code == 200
assert captured["data"]["client_id"] == "body-id"
assert captured["data"]["client_secret"] == "body-secret"
def test_400_when_neither_source_has_credentials(self, loopback_client, monkeypatch):
"""If body is empty AND env is empty, return 400 with a
friendly pointer to the API Keys panel — not a curt
"required" message and not a 500."""
monkeypatch.delenv("SENTINEL_CLIENT_ID", raising=False)
monkeypatch.delenv("SENTINEL_CLIENT_SECRET", raising=False)
# If the route ever calls requests.post here, the gate is broken
# — empty creds should never produce an outbound HTTP call.
fake = MagicMock(side_effect=AssertionError(
"requests.post should not be called when no credentials are configured"
))
with patch("requests.post", fake):
r = loopback_client.post(
"/api/sentinel/token",
data={},
headers={"Content-Type": "application/x-www-form-urlencoded"},
)
assert r.status_code == 400
detail = r.json().get("detail", "")
# The pointer to the API Keys panel is what makes this non-hostile.
assert "API Keys panel" in detail or "SENTINEL_CLIENT_ID" in detail
assert fake.call_count == 0
class TestSentinelTileCredResolution:
def test_env_fallback_when_body_omits_credentials(self, loopback_client, monkeypatch):
"""Tile route: no body credentials → uses env values."""
monkeypatch.setenv("SENTINEL_CLIENT_ID", "env-id")
monkeypatch.setenv("SENTINEL_CLIENT_SECRET", "env-secret")
token_resp = MagicMock()
token_resp.status_code = 200
token_resp.json = MagicMock(return_value={"access_token": "stub", "expires_in": 300})
process_resp = MagicMock()
process_resp.status_code = 200
process_resp.content = b"<png bytes>"
process_resp.headers = {"content-type": "image/png"}
captured: list = []
def fake_post(url, *args, **kwargs):
captured.append({"url": url, "data": kwargs.get("data"), "json": kwargs.get("json")})
if "openid-connect/token" in url:
return token_resp
return process_resp
with patch("requests.post", side_effect=fake_post):
r = loopback_client.post(
"/api/sentinel/tile",
json={
# Note: no client_id / client_secret in body
"preset": "TRUE-COLOR",
"date": "2026-01-01",
"z": 6, "x": 30, "y": 20,
},
)
assert r.status_code == 200
# First call was the token mint; verify it used env creds.
token_call = next(c for c in captured if "openid-connect/token" in c["url"])
assert token_call["data"]["client_id"] == "env-id"
assert token_call["data"]["client_secret"] == "env-secret"
def test_400_when_neither_source_has_credentials(self, loopback_client, monkeypatch):
monkeypatch.delenv("SENTINEL_CLIENT_ID", raising=False)
monkeypatch.delenv("SENTINEL_CLIENT_SECRET", raising=False)
fake = MagicMock(side_effect=AssertionError(
"requests.post should not be called when no credentials are configured"
))
with patch("requests.post", fake):
r = loopback_client.post(
"/api/sentinel/tile",
json={
"preset": "TRUE-COLOR",
"date": "2026-01-01",
"z": 6, "x": 30, "y": 20,
},
)
assert r.status_code == 400
detail = r.json().get("detail", "")
assert "API Keys panel" in detail or "SENTINEL_CLIENT_ID" in detail
assert fake.call_count == 0