mirror of
https://github.com/BigBodyCobain/Shadowbroker.git
synced 2026-05-28 10:01:31 +02:00
Shadowbroker
0fee36e8f7
Wikimedia's User-Agent policy asks API clients to identify themselves with a stable, contactable identifier so their operators can rate-limit or coordinate. Before this change, ShadowBroker was sending: - Backend (region_dossier.py): generic project default UA only; no Api-User-Agent. - Frontend (useRegionDossier.ts, WikiImage.tsx, NewsFeed.tsx): zero identifying header at all; three separate copy-pasted anonymous fetches with their own module-local caches. Three separate components doing the same broken thing meant policy fixes had to happen in three places, with no shared cache or kill switch. Fix (no UX change, zero hostility): == Backend == `backend/services/region_dossier.py` now sets explicit `User-Agent` + `Api-User-Agent` headers on every outbound Wikidata and Wikipedia request via a new `_WIKIMEDIA_REQUEST_HEADERS` constant. The identifier includes a contact path (issues page on the public GitHub repo). == Frontend == New shared helper `frontend/src/lib/wikimediaClient.ts`: - `fetchWikipediaSummary(title)` — single source of truth for Wikipedia REST summary lookups, with one shared LRU cache (in-flight requests deduplicated, 512-entry cap), `Api-User-Agent` on every fetch. - `fetchWikidataSparql(query)` — same shape for Wikidata SPARQL. - `WIKIMEDIA_API_USER_AGENT` — exported constant; one place to update if Wikimedia ever asks us to back off. Refactored three components to use the shared client: - `frontend/src/hooks/useRegionDossier.ts` — fetchLeader() and fetchLocalWikiSummary() now route through the shared helpers. - `frontend/src/components/WikiImage.tsx` — uses fetchWikipediaSummary, proper React state instead of module-mutation + forceUpdate trick. - `frontend/src/components/NewsFeed.tsx` — same shape. UX: byte-for-byte identical. Same thumbnails, same dossier content, same load behavior. The only observable difference is the outgoing request header. Note on #239 (route duplication): an audit-grade inventory shows 166 main.py routes are shadowed by router modules. That cleanup is too large to land safely in this PR; it will be staged as a separate ladder of small PRs grouped by router module. Tests: - `backend/tests/test_region_dossier_wikimedia_ua.py` — 3 tests asserting backend headers are present. - `frontend/src/__tests__/utils/wikimediaClient.test.ts` — 9 tests covering Api-User-Agent presence, shared cache, concurrent deduplication, disambiguation/HTTP-error/network-error fallthroughs, empty-input safety. Local: backend 76/76 security suite green, frontend 716/716 vitest suite green. Credit: tg12 (external security audit).
ShadowBroker Frontend
Next.js 16 dashboard with MapLibre GL, Cesium, and Framer Motion.
Development
npm install
npm run dev # http://localhost:3000
API URL Configuration
The frontend needs to reach the backend (default port 8000). Resolution order:
NEXT_PUBLIC_API_URLenv var — if set, used as-is (build-time, baked by Next.js)- Server-side (SSR) — falls back to
http://localhost:8000 - Client-side (browser) — auto-detects using
window.location.hostname:8000
Common scenarios
| Scenario | Action needed |
|---|---|
Local dev (localhost:3000 + localhost:8000) |
None — auto-detected |
LAN access (192.168.x.x:3000) |
None — auto-detected from browser hostname |
| Public deploy (same host, port 8000) | None — auto-detected |
Backend on different port (e.g. 9096) |
Set NEXT_PUBLIC_API_URL=http://host:9096 before build |
| Backend on different host | Set NEXT_PUBLIC_API_URL=http://backend-host:8000 before build |
Behind reverse proxy (e.g. /api path) |
Set NEXT_PUBLIC_API_URL=https://yourdomain.com before build |
Setting the variable
# Shell (Linux/macOS)
NEXT_PUBLIC_API_URL=http://myserver:8000 npm run build
# PowerShell (Windows)
$env:NEXT_PUBLIC_API_URL="http://myserver:8000"; npm run build
# Docker Compose (set in .env file next to docker-compose.yml)
NEXT_PUBLIC_API_URL=http://myserver:8000
Note: This is a build-time variable. Changing it requires rebuilding the frontend.
Theming
Dark mode is the default. A light/dark toggle is available in the left panel toolbar.
Theme preference is persisted in localStorage as sb-theme and applied via
data-theme attribute on <html>. CSS variables in globals.css define all
structural colors for both themes.