Reported by @tg12. Pre-fix, two problems lived on the GET endpoint:
1. `GET /api/ai/connect-info?reveal=true` returned the full HMAC
secret in the response body on every Connect modal open. Even
gated to require_local_operator, that put the secret into
browser history, dev-tools network panels, browser disk caches,
HAR exports, and screen captures.
2. The same GET endpoint auto-bootstrapped (generated + persisted)
the secret on a mere read. Side effects on a GET are a footgun:
browser prefetchers, mirror tools, and casual curl-from-history
would all silently mint+persist a fresh secret.
Backend (backend/routers/ai_intel.py)
-------------------------------------
GET /api/ai/connect-info — always returns the MASKED
fingerprint (first6 + bullets
+ last4). No `?reveal` param.
NO auto-bootstrap. When the
secret is missing, returns
`hmac_secret_set: false` and
tells the caller to POST to
/bootstrap.
POST /api/ai/connect-info/bootstrap — NEW. Mints+persists the secret
if missing. Idempotent. Never
returns the full secret in the
response body.
POST /api/ai/connect-info/reveal — NEW. Returns the full secret
with Cache-Control: no-store,
no-cache, must-revalidate +
Pragma: no-cache + Expires: 0.
POST so the body never lands
in URL history. 404 (with a
pointer to /bootstrap) when
the secret isn't set.
POST /api/ai/connect-info/regenerate — keeps existing one-time-reveal
behavior (regen IS a deliberate
destructive action triggered
by the operator). Same
no-store/no-cache headers added
so even the regen response
doesn't get cached.
Frontend (AIIntelPanel.tsx, OnboardingModal.tsx)
------------------------------------------------
* On mount: GET (masked only). If hmac_secret_set: false, fire a
transparent POST /bootstrap and refresh the masked fingerprint.
Operator sees no behavior change from pre-#302.
* Reveal (eye icon): lazy POST /reveal — secret only travels when
the operator explicitly clicks the button.
* Copy: lazy POST /reveal too — copying without a prior reveal
works exactly like before, just routed through the new endpoint.
* Regenerate: POST returns the new secret (same as before, but the
response now has no-store headers).
* The displayed snippet uses the masked fingerprint until the
operator clicks Reveal or Copy.
Tests (backend/tests/test_openclaw_connect_info_reveal.py — 13 tests)
---------------------------------------------------------------------
* GET returns masked + the full secret never appears in r.text
* GET does NOT auto-bootstrap when missing
* GET silently ignores any ?reveal=true query (back-compat noise)
* POST /bootstrap mints when missing, idempotent when set
* POST /bootstrap never returns the full secret
* POST /reveal returns the full secret with Cache-Control: no-store,
no-cache + Pragma: no-cache + Expires: 0
* POST /reveal 404s with a pointer to /bootstrap when no secret
* POST /regenerate returns the new secret with the same headers
* Anonymous remote callers get 403 on ALL FOUR endpoints (parametric
regression against the same allowlist used elsewhere).
Adjacent suites still green: test_openclaw_route_security,
test_no_new_duplicate_routes, test_control_surface_auth. 67/67 pass
locally.
Credit: @tg12 for the audit report.
== Per-install operator handle for every third-party API call ==
Before this PR, every Shadowbroker install identified itself to
Wikipedia, Wikidata, Nominatim, GDELT, OpenMHz, Broadcastify,
weather.gov, NUFORC, Sentinel/Planetary Computer, TinyGS / CelesTrak,
Shodan, Finnhub, and others with a single project-wide User-Agent
("Shadowbroker/1.0" or "ShadowBroker-OSINT/1.0"). From the upstream's
perspective every install in the world looked like one giant scraper.
If one install misbehaved, the upstream's only recourse was to block
"Shadowbroker" as a whole.
PR #284 inadvertently doubled down on this in the frontend by
introducing a shared `WIKIMEDIA_API_USER_AGENT` constant. This PR
retrofits both backends to per-operator attribution.
New setting: OPERATOR_HANDLE (env var / settings UI / auto-gen)
New helper: network_utils.outbound_user_agent("purpose")
The handle is auto-generated as "operator-XXXXXX" on first call (the
"shadow-" prefix from earlier drafts was deliberately dropped — too
suspicious-looking for abuse-detection systems). Operators can
override via OPERATOR_HANDLE; the value is sanitized to lowercase
alphanumeric+dash+underscore and capped at 48 chars. Persisted to
backend/data/operator_handle.json so it survives container restarts.
Retrofitted call sites (every previously-MONSTER User-Agent):
- services/region_dossier.py (Wikipedia + Wikidata + Nominatim)
- services/geocode.py (Nominatim)
- services/sentinel_search.py (Microsoft Planetary Computer)
- services/feed_ingester.py (operator-curated RSS feeds)
- services/fetchers/earth_observation.py (weather.gov, NUFORC)
- services/fetchers/infrastructure.py
- services/fetchers/aircraft_database.py
- services/fetchers/route_database.py
- services/fetchers/trains.py
- services/fetchers/meshtastic_map.py
- services/shodan_connector.py
- services/unusual_whales_connector.py (Finnhub)
- services/tinygs_fetcher.py (CelesTrak + TinyGS)
- services/sar/sar_products_client.py
- services/geopolitics.py (GDELT)
- services/radio_intercept.py (Broadcastify + OpenMHz)
- routers/cctv.py + main.py (CCTV proxy)
- routers/ai_intel.py
- scripts/convert_power_plants.py (release-time data refresh)
Spoofed browser UAs removed (issues #289 / #290 / #291 — tg12 audit):
- cloudscraper-based Chrome impersonation against api.openmhz.com
-> replaced with honest requests + per-install UA
- Mozilla/5.0 spoofed UA on Broadcastify scrape
-> replaced with honest UA
- Mozilla/5.0 + fake first-party Referer on OpenMHz audio relay
-> replaced with honest UA
- cloudscraper dependency dropped from pyproject.toml + uv.lock
Frontend retrofit:
- new GET /api/settings/operator-handle endpoint (local-operator
gated) returns the install's handle
- frontend/src/lib/wikimediaClient.ts fetches the handle once on
first use, caches it for page lifetime, embeds it in the
Api-User-Agent for every Wikipedia / Wikidata browser-direct call
== GDELT GCS-direct fix ==
GDELT's data.gdeltproject.org is a CNAME to a Google Cloud Storage
bucket. GCS responds with the wildcard *.storage.googleapis.com cert
which legitimately does NOT cover the GDELT custom domain, so Python's
TLS verification correctly refuses the connection. Some networks
happen to route through a path where this works; many (notably Docker
Desktop's outbound NAT on local installs) do not. Verified on the
maintainer's local install: GDELT was unreachable; 1610 geopolitical
events / 48 export files were dropping silently.
Fix: services/geopolitics._gcs_direct_gdelt_url() rewrites any
data.gdeltproject.org URL to its GCS-direct equivalent
(storage.googleapis.com/data.gdeltproject.org/...) where the standard
GCS cert is genuinely valid. api.gdeltproject.org and every other host
are left untouched.
Confirmed live: backend log goes from
GDELT lastupdate failed: 500
to
Downloading 48 GDELT export files...
Downloaded 48/48 GDELT exports
GDELT parsed: 1610 conflict locations from 48 files
== Tests ==
backend/tests/test_per_operator_outbound_attribution.py (12 tests)
backend/tests/test_gdelt_gcs_direct_rewrite.py (6 tests)
backend/tests/test_region_dossier_wikimedia_ua.py (updated to
pin the helper + per-operator handle, not the old constant)
frontend/src/__tests__/utils/wikimediaClient.test.ts (rewritten
to mock /api/settings/operator-handle and assert per-operator UA)
Local: backend 114/114 security+audit+round7a suite green;
frontend 718/718 vitest suite green.
Credit: tg12 (external security audit, issues #289/#290/#291
relating to spoofed UAs); BigBodyCobain (operator-prefix call,
GDELT cloud-vs-local diagnosis).
Ship the v0.9.79 runtime refresh with transport lane isolation, Infonet secure-message address management, MeshChat MQTT controls, selected asset trail behavior, telemetry panel refinements, onboarding updates, and desktop/package metadata alignment.
Also ignore local graphify work products so analysis folders do not leak into future commits.
BigBodyCobain