Auto-relay connect DMs with End Contact severing, signed fleet prekey lookup,
OpenClaw private Infonet channel intents, headless relay Tor bootstrap on redeploy,
and swarm/DM live verification scripts.
Co-authored-by: Cursor <cursoragent@cursor.com>
Seed relay nodes should track CI-published backend images instead of local builds that fail without full monorepo context.
Co-authored-by: Cursor <cursoragent@cursor.com>
Ship sb-testnet fleet defaults, swarm/join API, NODE launcher registration step, and meshnode script defaults so users discover peers via the signed seed manifest without manual peer lists.
Co-authored-by: Cursor <cursoragent@cursor.com>
Signed peer manifest pull/announce on the seed, immediate hashchain push for gate messages, seed-only Docker defaults, and stale-genesis sync diagnostics.
Co-authored-by: Cursor <cursoragent@cursor.com>
Update gate-resync decomposition expectations for Infonet embed and harden GateView stream snapshot waits for slower CI runners.
Co-authored-by: Cursor <cursoragent@cursor.com>
Rename Mesh Chat to Meshtastic Chat, embed the Infonet terminal with Arti/Tor warmup, improve the agent shell PTY (git in the backend image, operator PATH), and add docker-compose.override for local image builds. Gitignore Hermes Agent runtime installs.
Co-authored-by: Cursor <cursoragent@cursor.com>
Uses a local-operator WebSocket bash session, keeps the map interactive, and SNAP docks the shell back into Mesh Chat instead of a floating blurred panel.
Co-authored-by: Cursor <cursoragent@cursor.com>
Anchors to the Mesh Chat box, stretches on tab enter, and supports user resize without changing the fixed left column width.
Co-authored-by: Cursor <cursoragent@cursor.com>
Carry SHA-256 hashes for the source zip, MSI, and setup EXE into release_digests.json while retaining prior release entries.
Co-authored-by: Cursor <cursoragent@cursor.com>
Align backend, desktop, helm, and frontend package versions for the Telegram OSINT and OpenClaw recon release.
Co-authored-by: Cursor <cursoragent@cursor.com>
Add Telegram OSINT with hourly incremental t.me scraping, metro geocoding
separate from news centroids, threat-intercept popup UI with inline media,
and HTML markers above alert boxes so pins stay clickable. Expose GFW_API_TOKEN
in onboarding and Settings Maritime; harden GFW/CCTV/geo fetchers. Port Osiris-
derived recon, SCM, entity graph, malware/cyber feeds, sanctions, and submarine
cable layers with tests and documentation.
Co-authored-by: Cursor <cursoragent@cursor.com>
Align full /api/live-data with slow-tier orjson options, remove dead main.py duplicate, cap slow batches to pool size, cancel queued work on timeout, and stop retrying HTTP 4xx/5xx.
Co-authored-by: Cursor <cursoragent@cursor.com>
Default python main.py to loopback, deep-copy dashboard snapshots outside the store lock with ETag on full live-data, and route GDELT/LiveUAMap/CCTV/slow-tier work through an isolated executor so Playwright jobs cannot starve fast-tier workers.
Co-authored-by: Cursor <cursoragent@cursor.com>
Operators enable Polymarket/Kalshi correlation from Global Threat Intercept with a consent dialog; polls use a jittered schedule separate from the slow tier. Right-click Sentinel imagery returns up to three signed scenes again.
Co-authored-by: Cursor <cursoragent@cursor.com>
Each install pulls ~60-day sightings from nuforc.org every Monday; disk cache
matches weekly cadence so users keep current pins between restarts.
Co-authored-by: Cursor <cursoragent@cursor.com>
Filter stale rows out of nuforc_recent_sightings.json on load; add requests-based
live scrape when curl is disabled; daily scheduler rebuild instead of weekly-only.
Co-authored-by: Cursor <cursoragent@cursor.com>
- User-Agent is per-install handle only (no Shadowbroker product token)
- LiveUAMap: Windows UI consent when enabling Global Incidents; env override
- Meshtastic callsign upstream header off by default (opt-in true)
- Expanded docs/OUTBOUND_DATA.md and README link for CCTV, basemap, Broadcastify
Co-authored-by: Cursor <cursoragent@cursor.com>
Operators can set DEEPSTATE_MIRROR_COMMIT for immutable frontline ingest; Madrid KML tries HTTPS then HTTP without changing camera image URLs or proxy Referers.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(ci): switch mirror-to-github job from PAT to per-repo SSH deploy key
GitHub fine-grained PATs are capped at 366 days, classic PATs would
need 'public_repo' (broader scope than needed). Per-repo SSH deploy
keys are tighter:
- Can ONLY push to BigBodyCobain/Shadowbroker (no access to anything
else, not even other repos owned by the same account).
- Never expire.
- Rotating == one-click delete on github.com/.../settings/keys.
Changes:
- New CI/CD variable GITHUB_MIRROR_SSH_KEY (File, Protected) holding
the ed25519 private half. Public half lives on the repo's deploy
keys with write access enabled.
- mirror-to-github before_script writes the key to ~/.ssh/id_ed25519,
pins github.com host fingerprints (ed25519 + ecdsa + rsa from the
2023-03-24 rotation) into ~/.ssh/known_hosts so we never trust a
MITM, then pushes via git@github.com:... instead of HTTPS.
- Job rule now gates on GITHUB_MIRROR_SSH_KEY (the new var) instead
of GITHUB_MIRROR_TOKEN (which never existed).
After this lands, every commit pushed directly to GitLab main will
mirror back to GitHub main automatically — closing the loop on
bi-directional sync.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(secret-scan): exempt SSH known_hosts entries from leaked-key detection
PR #331 introduced github.com host fingerprints pinned in
.gitlab-ci.yml's mirror-to-github before_script. The scanner flagged
them as embedded secrets and blocked CI:
BLOCKED: Embedded secrets/tokens found in:
.gitlab-ci.yml
133: github.com ssh-ed25519 AAAA...
135: github.com ssh-rsa AAAA...
These are PUBLIC host keys — the whole point of pinning known_hosts is
to publish the fingerprint widely so a MITM is detectable. They are
documented at https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/githubs-ssh-key-fingerprints
and committing them is the correct, secure practice.
Fix: add a KNOWN_HOSTS_LINE regex to the content-scan block that
recognizes `<host-or-ip> [salt] <algo> AAAA...` shape lines (the
exact format used in ~/.ssh/known_hosts) and filters them out before
flagging the file. Bare `ssh-rsa AAAA...` lines without a host prefix
are still caught — only the host-key shape is exempt.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
The build-backend and build-frontend jobs were failing immediately after
identity verification finally allocated runners:
$ docker buildx create --use --name multiarch --driver docker-container
ERROR: could not create a builder instance with TLS data loaded from
environment. Please use `docker context create <context-name>` to create
a context for current environment and then create a builder instance
with context set to <context-name>
The dind service exports DOCKER_HOST=tcp://docker:2376 +
DOCKER_TLS_CERTDIR=/certs, but buildx --driver docker-container doesn't
read TLS from those env vars directly. Documented GitLab fix: create an
empty `docker context` (which inherits the current TLS env), then bind
buildx to that context name as a positional arg.
After this lands, the multi-arch buildx jobs should actually build and
push amd64 + arm64 images to
registry.gitlab.com/bigbodycobain/shadowbroker/backend:latest
registry.gitlab.com/bigbodycobain/shadowbroker/frontend:latest
Surfaced by the post-verification pipeline at
https://gitlab.com/bigbodycobain/Shadowbroker/-/pipelines/2550501798
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Pipelines on the GitLab mirror have been instant-failing with 0 jobs and
no started_at since the project was created — classic "shared runners
not allocated to unverified free-tier accounts" pattern. The account is
now identity-verified; this trivial comment bump exists solely to fire a
fresh pipeline that confirms runners now pick up the build-backend and
build-frontend jobs.
If the resulting pipeline produces real jobs that build the multi-arch
images and push them to registry.gitlab.com/bigbodycobain/shadowbroker/{backend,frontend},
the GitLab install path is at full parity with the GitHub one.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Re-cut v0.9.81 binaries from current main (which now includes the
private gate + DM hashchain spool from #326 and the gate-directory
test from #327). All three artifacts were signed with the same
minisign updater key as the original v0.9.81 release, so existing
v0.9.81 installs on Tauri auto-update accept the new bundles.
Updated hashes (verified against released assets):
- ShadowBroker_v0.9.81.zip f81f454bdc88e9a32c351df38212b8cfa624704d65764b971bb091eef62259c6
- ShadowBroker_0.9.81_x64-setup.exe 25e9a95d0d8ce959a7d08fe8e7406772ae24b596652793e81d1de5d02510a5a6
- ShadowBroker_0.9.81_x64_en-US.msi 34e655fc0c0f195ee4ac978f228a4b2b9d5565253b8771aca9ef4693409e9e70
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Adds the focused test Codex wrote alongside the gate-directory UI work
that already shipped in #326 (the `renderGateDirectory` helper used
both under the Infonet logo on the landing screen and as the output of
the `gates` command in the terminal).
The renderer itself is already on origin/main; this PR just ships the
test so CI catches regressions to the dual-variant render.
Verified locally:
- frontend npm run test:ci -- src/__tests__/mesh/infonetShellGateDirectory.test.tsx → 1/1 pass
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Private gate messages and offline DMs now ride the Infonet hashchain
as ciphertext-only events, replicated across nodes via private
transports (Tor onion / RNS / loopback) and decrypted only by parties
holding the gate or recipient keys.
Hashchain core (mesh_hashchain.py)
----------------------------------
* New ``append_private_gate_message`` and ``append_private_dm_message``
append paths with full signature verification, public-key binding,
revocation check, and replay protection in a dedicated sequence
domain (so a gate post does not consume the author's public broadcast
sequence, and a DM cannot replay-block a public message at sequence=1).
* Fork validation and full-chain validation now accept the gate
signature compatibility variants — older signatures that canonicalize
with/without epoch or reply_to still verify, so a re-sync from an
older peer doesn't reject still-valid history.
* DM hashchain spool: capped at 2 active sealed offline DMs per
recipient mailbox, plus a per-(sender, recipient) cap so one prolific
sender can't consume both slots. 1-hour TTL on the cap counter.
Spool intentionally small — it's an offline bootstrap channel,
not a persistent mailbox.
* Rebuild-state preserves the gate sequence domain across reloads so
a chain reload doesn't accidentally let an old gate sequence
replay-collide on next append.
Schema enforcement (mesh_schema.py)
-----------------------------------
* Private gate + DM payloads have closed allowlists of fields.
Plaintext keys (``message``, ``plaintext``, ``_local_plaintext``,
``_local_reply_to``) are explicit rejection-bait — they raise before
the event ever touches the chain.
* DM ciphertext + nonce must look like base64-ish sealed bytes;
obvious base64-encoded plaintext shapes are rejected.
* ``transport_lock`` required: DM hashchain spool requires
``private_strong``; gate accepts ``private``/``private_strong``/
``rns``/``onion``.
Defense-in-depth at the network layer (main.py + mesh_public.py)
----------------------------------------------------------------
* ``_infonet_sync_response_events`` now silently redacts private events
(gate_message + dm_message) unless the request looks like a loopback /
onion / RNS / private transport caller. If an operator accidentally
exposes :8000 to the public internet, an external puller gets
public events only — never ciphertext.
* ``_sync_from_peer`` raises ``PeerSyncRateLimited`` for 429 (handled
as 4-tuple return with retry_after_s) and ``PeerSyncHTTPError`` for
other non-200 statuses (handled by ``_run_public_sync_cycle`` to
honor server cooldown hints even outside the 429 path).
DM relay hydration (main.py)
-----------------------------
* New ``_hydrate_dm_relay_from_chain``: when accepted dm_message chain
events arrive on a node, they get deposited into the local DM relay
store with a deterministic sender_token_hash so re-sync of the same
event is idempotent. Recipients see the ciphertext as a normal DM
on their next poll and decrypt with their existing recipient key.
Other surfaces
--------------
* meshnode.bat / meshnode.sh now set ``MESH_INFONET_ALLOW_CLEARNET_SYNC=
false`` and the participant runtime flags by default so a freshly
spun-up node defaults to private-only sync.
* InfonetTerminal/InfonetShell.tsx adds a gate directory renderer for
the new private-gate workflow.
* docker-compose.relay.yml binds the relay backend to 127.0.0.1:8000
only; Tor's hidden service forwards onion traffic into 127.0.0.1.
Public clearnet :8000 stays off the network edge.
Tests
-----
* 7 new tests in test_private_gate_hashchain.py + test_private_dm_
hashchain.py covering: gate fork accepts ciphertext propagation,
gate fork rejects plaintext, append rejects plaintext before
normalize, append requires private_strong, append rejects
non-sealed ciphertext shape, DM spool 2-per-recipient + 1-per-pair
cap, DM hydration delivers to poll/claim.
* Updated test_mesh_node_bootstrap_runtime.py covers 429 backoff via
PeerSyncRateLimited 4-tuple AND PeerSyncHTTPError exception.
* Updated test_s14b_public_sync_gate_filter.py + test_s9b_gate_store_
hydration.py + test_gate_write_cutover.py cover the new private
redaction on public sync responses.
* test_private_gate_hashchain.py + test_private_dm_hashchain.py:
10 passed locally.
* Combined mesh-relevant suite (the 5 modified existing tests +
2 new): 17 passed.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* fix(start-scripts): find bundled privacy_core.dll next to script
start.bat and start.sh only checked the source-tree DLL path
(``privacy-core/target/release/privacy_core.dll``), not the bundled
location where MSI/AppImage/DMG installers stage the library directly
next to the script in backend-runtime/.
Users running start.bat from inside an MSI install dir (a documented
workaround when the desktop shell crashes) saw a scary "install Rust"
warning even though the DLL was sitting right next to them. See issue
#319 for the user-reported confusion.
Fix: add a fallback check for the bundled location before falling
through to the "build privacy-core from source" warning. Source-tree
behavior unchanged — the source path is still preferred when present.
Also re-stamps the v0.9.81 source archive: ``release_digests.json``
v0.9.81 zip hash updated to point at the rebuilt source archive that
contains these script changes. MSI/EXE/sig hashes are unchanged (the
scripts live at the repo root, not inside the desktop bundle).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(#319): bundle start.bat + start.sh into the MSI/EXE installers
Follow-up to the start-script DLL fallback fix in the prior commit.
ChrisMTheMan's report on #319 made it clear the workaround flow was:
1. MSI install crashes on launch (different bug, fixed in v0.9.81)
2. User goes looking for start.bat to launch the backend manually
3. start.bat isn't in their install dir, so they go fetch it from GitHub
4. They get a working script but it doesn't know about the bundled
privacy_core.dll layout, so they see a scary "install Rust" warning
The prior commit fixed step 4. This commit fixes step 3 — start.bat and
start.sh now ship inside the MSI/EXE installers (staged into
backend-runtime/ next to the privacy_core.dll they expect to find).
After the rebuild lands, an MSI user looking for these scripts finds
them right inside their install dir, already pointing at the correct
bundled DLL location.
What changed
------------
* ``build-backend-runtime.cjs`` now has a ``stageStartScripts()`` step
that copies start.bat and start.sh from the repo root into the
staged backend-runtime/. Preserves the executable bit on .sh under
POSIX.
* ``release_digests.json`` v0.9.81 block hashes refreshed for the
rebuilt MSI / EXE / source-zip (the scripts being bundled changed
the MSI/EXE contents; the source zip also includes the start-script
fix from the prior commit).
ShadowBroker_v0.9.81.zip 6.06 MB
af8c87ccdece8fbb9aadc6be63cce10d3fcba74e6d87ef83289dda6d555fd270
ShadowBroker_0.9.81_x64_en-US.msi 122.4 MB
8977c9a1c54e1f0d030436be9c4e3d81d766cc0080699eb747649095f360c7ff
ShadowBroker_0.9.81_x64-setup.exe 76.5 MB
4e866fa0423c0c2470ed32f4809167a7815dc23ee7762b69e95681c1f3a28250
Post-merge plan
---------------
Force-move the v0.9.81 tag to this commit and replace ALL release
assets on the GitHub release: zip, msi, exe, both .sig files,
latest.json, SHA256SUMS.txt, release-manifest.json.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
What this release does
----------------------
1. Establishes a fresh Tauri updater signing keypair. The previous keypair
(pubkey baked into v0.9.79 / v0.9.8) had no matching private key on
any maintainer-controlled machine — every prior release shipped
without signatures, so auto-update has never actually worked. v0.9.81
rotates to a new pubkey and ships signed installers + latest.json so
every release from here is a one-click upgrade.
2. Fixes the ``admin_session_required`` race in TopRightControls.tsx.
The updateAction state used to default to ``auto_apply`` at React-init
time. A click on the Update button before the async runtime probe
completed went down the auto_apply path (POST /api/system/update),
which throws ``admin_session_required`` on fresh sessions. Desktop
installs now default to ``manual_download`` based on synchronous
``window.__TAURI__`` detection at useState init.
One-time cost for current installs
----------------------------------
Anyone on v0.9.79 or v0.9.8 will see the in-app Update button still
trigger the broken path on their existing install (the fix only takes
effect once they're ON v0.9.81). The MANUAL DOWNLOAD button in the
update dialog opens the GitHub release page, where they grab the .msi
and run it. After that one manual hop, all future updates are seamless.
Release artifacts
-----------------
ShadowBroker_v0.9.81.zip 6.06 MB
42f8a51f9a5690d1e7349d90d8ecf2d163c9061d6cf90c69ee03647a785437ff
ShadowBroker_0.9.81_x64_en-US.msi 122.4 MB
a45b177c26c95d2b28d71592d7147e88ff4e104865f214fde11249d311ec9e25
ShadowBroker_0.9.81_x64-setup.exe 76.5 MB
eca884b9d37eeccd0f11c91dcc6f6ae1b3609d9dee72bd73c37c9a427babfef2
Plus .sig files for the .msi and .exe, plus a signed latest.json for
the Tauri updater endpoint.
Sizes match the v0.9.79 / v0.9.8 reference shape within drift for
the new TopRightControls patch.
release_digests.json keeps v0.9.79 + v0.9.8 blocks alongside v0.9.81
so operators still on those versions continue to validate cleanly
during the rollout transition.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Issues #319 and #296 reported that the installed v0.9.79 Windows MSI/EXE
crashed on launch with:
thread 'main' panicked ... failed to setup app: error encountered
during setup hook: ShadowBroker cannot start: the bundled local
backend failed to launch.
technical detail: managed_backend_exited_early:exit code: 103
Root cause: ``backend/pyproject.toml`` declares ``defusedxml>=0.7.1`` and
``PySocks==1.7.1`` as runtime dependencies, but the venv used to build
v0.9.79 (and the initial v0.9.8 publish) had both missing. When
``services/fetchers/aircraft_database.py`` does
``import defusedxml.ElementTree`` at startup, Python raises
``ModuleNotFoundError`` and uvicorn exits, which Tauri reports as
``managed_backend_exited_early``.
Both packages now installed in the build venv. ``main.py`` imports
end-to-end with only the expected ``plane_alert_db.json not found``
warning (runtime-state file, populated on first launch).
Rebuilt artifacts on the maintainer's local machine:
ShadowBroker_v0.9.8.zip 6.06 MB
183bb5cd62b9b9349d95df5ef7696cb6ca810ab4b991fa9dab6f898af4c7a175
ShadowBroker_0.9.8_x64_en-US.msi 122.4 MB
fe22f9d51e4360d74c18a7250c2fbb9ed4fa4c7a884b3ac0d04a21115466386b
ShadowBroker_0.9.8_x64-setup.exe 76.5 MB
94a0309862e9c81c92cdcbfea8eec9dbb97eef19ded82b26217b397defbc810c
After this merges, the v0.9.8 tag will be force-moved to this commit and
the GitHub release assets replaced so the integrity chain validates
against the working installers instead of the broken ones.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Bumps every hardcoded 0.9.79 → 0.9.8 across backend, frontend,
desktop-shell, helm, lockfiles, test fixtures. Refreshes the in-app
ChangelogModal HEADLINE_FEATURES, NEW_FEATURES, and BUG_FIXES with the
v0.9.8 highlights.
Release artifacts built locally and hashed into release_digests.json:
ShadowBroker_v0.9.8.zip 6.06 MB
d506f6b8462ccb12096f0cd9462233be58928094240416b65fb3127bdd1f3820
ShadowBroker_0.9.8_x64_en-US.msi 122.4 MB
d4be4cb68c3e6409fff54c225acdcdd08e27d5d6d2b31616d78d2a4f6812991d
ShadowBroker_0.9.8_x64-setup.exe 76.5 MB
1115d1f5cf37edd03ea2c21d821c7626e1bf3319c990402aaa0293bca46fea67
Sizes match the v0.9.79 reference shape (5.76 MB / 117 MB / 72.9 MB)
within expected drift for new code. The .zip is a `git archive` of the
v0.9.8 source tree (matching v0.9.79's approach).
Audit confirms no .env, .key, .venv-dir, or cache files leaked into the
backend-runtime bundle. Python 3.11.9 + 199 site-packages + privacy_core
all staged correctly.
Headline changes since v0.9.79:
* Cumulative fuel/CO2 per flight (#317) — running totals since first
observation, not just per-hour rate.
* AIS maritime resilience (#314, #316) — outage banner + AISHub REST
fallback when AISStream WebSocket primary is offline.
* Data-layer repair (#311, #312) — UAP fallback respects the 60-day
cutoff; GPS jamming threshold tuning + nac_p=0 inclusion so the layer
actually fires.
* Per-flight source attribution (#313) — source field on every record.
* Cross-node DM mailbox replication (#309).
* Infonet sync HTTP 429 honored (#310).
Test fixtures updated:
* test_per_operator_outbound_attribution.py — added v0.9.8 UA strings
to the banned-aggregate-literals list (alongside v0.9.79).
* updateRuntime.test.ts — bumped asset filename fixtures to v0.9.8.
release_digests.json keeps the v0.9.79 block alongside v0.9.8 so
operators still on 0.9.79 validate cleanly during the rollout.
The accent narrowing fix in ChangelogModal (one feature uses 'purple',
two use 'cyan' so the renderer's `accent === 'purple'` comparison
still type-checks) is included.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
BigBodyCobain