MLS export/reset and accept use live HTTP so uvicorn privacy-core state stays consistent; relay persistence and sender_seal fixes enable invite-accept-shared decrypt across onion peers. Adds participant/e2e compose overlays and harness recovery with optional Tor-only replicate mode.
Co-authored-by: Cursor <cursoragent@cursor.com>
Retry announce/manifest while Tor circuits warm on NODE and startup bootstrap.
Add verify_swarm_fresh_participant.py for empty-volume GHCR smoke tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
fetchDmPublicKey now passes allowLegacyAgentId: false for short-address
contact requests; update the assertion to match the new call signature.
Co-authored-by: Cursor <cursoragent@cursor.com>
Auto-relay connect DMs with End Contact severing, signed fleet prekey lookup,
OpenClaw private Infonet channel intents, headless relay Tor bootstrap on redeploy,
and swarm/DM live verification scripts.
Co-authored-by: Cursor <cursoragent@cursor.com>
Seed relay nodes should track CI-published backend images instead of local builds that fail without full monorepo context.
Co-authored-by: Cursor <cursoragent@cursor.com>
Ship sb-testnet fleet defaults, swarm/join API, NODE launcher registration step, and meshnode script defaults so users discover peers via the signed seed manifest without manual peer lists.
Co-authored-by: Cursor <cursoragent@cursor.com>
Signed peer manifest pull/announce on the seed, immediate hashchain push for gate messages, seed-only Docker defaults, and stale-genesis sync diagnostics.
Co-authored-by: Cursor <cursoragent@cursor.com>
Update gate-resync decomposition expectations for Infonet embed and harden GateView stream snapshot waits for slower CI runners.
Co-authored-by: Cursor <cursoragent@cursor.com>
Rename Mesh Chat to Meshtastic Chat, embed the Infonet terminal with Arti/Tor warmup, improve the agent shell PTY (git in the backend image, operator PATH), and add docker-compose.override for local image builds. Gitignore Hermes Agent runtime installs.
Co-authored-by: Cursor <cursoragent@cursor.com>
Uses a local-operator WebSocket bash session, keeps the map interactive, and SNAP docks the shell back into Mesh Chat instead of a floating blurred panel.
Co-authored-by: Cursor <cursoragent@cursor.com>
Anchors to the Mesh Chat box, stretches on tab enter, and supports user resize without changing the fixed left column width.
Co-authored-by: Cursor <cursoragent@cursor.com>
Carry SHA-256 hashes for the source zip, MSI, and setup EXE into release_digests.json while retaining prior release entries.
Co-authored-by: Cursor <cursoragent@cursor.com>
Align backend, desktop, helm, and frontend package versions for the Telegram OSINT and OpenClaw recon release.
Co-authored-by: Cursor <cursoragent@cursor.com>
Add Telegram OSINT with hourly incremental t.me scraping, metro geocoding
separate from news centroids, threat-intercept popup UI with inline media,
and HTML markers above alert boxes so pins stay clickable. Expose GFW_API_TOKEN
in onboarding and Settings Maritime; harden GFW/CCTV/geo fetchers. Port Osiris-
derived recon, SCM, entity graph, malware/cyber feeds, sanctions, and submarine
cable layers with tests and documentation.
Co-authored-by: Cursor <cursoragent@cursor.com>
Align full /api/live-data with slow-tier orjson options, remove dead main.py duplicate, cap slow batches to pool size, cancel queued work on timeout, and stop retrying HTTP 4xx/5xx.
Co-authored-by: Cursor <cursoragent@cursor.com>
Default python main.py to loopback, deep-copy dashboard snapshots outside the store lock with ETag on full live-data, and route GDELT/LiveUAMap/CCTV/slow-tier work through an isolated executor so Playwright jobs cannot starve fast-tier workers.
Co-authored-by: Cursor <cursoragent@cursor.com>
Operators enable Polymarket/Kalshi correlation from Global Threat Intercept with a consent dialog; polls use a jittered schedule separate from the slow tier. Right-click Sentinel imagery returns up to three signed scenes again.
Co-authored-by: Cursor <cursoragent@cursor.com>
Each install pulls ~60-day sightings from nuforc.org every Monday; disk cache
matches weekly cadence so users keep current pins between restarts.
Co-authored-by: Cursor <cursoragent@cursor.com>
Filter stale rows out of nuforc_recent_sightings.json on load; add requests-based
live scrape when curl is disabled; daily scheduler rebuild instead of weekly-only.
Co-authored-by: Cursor <cursoragent@cursor.com>
- User-Agent is per-install handle only (no Shadowbroker product token)
- LiveUAMap: Windows UI consent when enabling Global Incidents; env override
- Meshtastic callsign upstream header off by default (opt-in true)
- Expanded docs/OUTBOUND_DATA.md and README link for CCTV, basemap, Broadcastify
Co-authored-by: Cursor <cursoragent@cursor.com>
Operators can set DEEPSTATE_MIRROR_COMMIT for immutable frontline ingest; Madrid KML tries HTTPS then HTTP without changing camera image URLs or proxy Referers.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(ci): switch mirror-to-github job from PAT to per-repo SSH deploy key
GitHub fine-grained PATs are capped at 366 days, classic PATs would
need 'public_repo' (broader scope than needed). Per-repo SSH deploy
keys are tighter:
- Can ONLY push to BigBodyCobain/Shadowbroker (no access to anything
else, not even other repos owned by the same account).
- Never expire.
- Rotating == one-click delete on github.com/.../settings/keys.
Changes:
- New CI/CD variable GITHUB_MIRROR_SSH_KEY (File, Protected) holding
the ed25519 private half. Public half lives on the repo's deploy
keys with write access enabled.
- mirror-to-github before_script writes the key to ~/.ssh/id_ed25519,
pins github.com host fingerprints (ed25519 + ecdsa + rsa from the
2023-03-24 rotation) into ~/.ssh/known_hosts so we never trust a
MITM, then pushes via git@github.com:... instead of HTTPS.
- Job rule now gates on GITHUB_MIRROR_SSH_KEY (the new var) instead
of GITHUB_MIRROR_TOKEN (which never existed).
After this lands, every commit pushed directly to GitLab main will
mirror back to GitHub main automatically — closing the loop on
bi-directional sync.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(secret-scan): exempt SSH known_hosts entries from leaked-key detection
PR #331 introduced github.com host fingerprints pinned in
.gitlab-ci.yml's mirror-to-github before_script. The scanner flagged
them as embedded secrets and blocked CI:
BLOCKED: Embedded secrets/tokens found in:
.gitlab-ci.yml
133: github.com ssh-ed25519 AAAA...
135: github.com ssh-rsa AAAA...
These are PUBLIC host keys — the whole point of pinning known_hosts is
to publish the fingerprint widely so a MITM is detectable. They are
documented at https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/githubs-ssh-key-fingerprints
and committing them is the correct, secure practice.
Fix: add a KNOWN_HOSTS_LINE regex to the content-scan block that
recognizes `<host-or-ip> [salt] <algo> AAAA...` shape lines (the
exact format used in ~/.ssh/known_hosts) and filters them out before
flagging the file. Bare `ssh-rsa AAAA...` lines without a host prefix
are still caught — only the host-key shape is exempt.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
The build-backend and build-frontend jobs were failing immediately after
identity verification finally allocated runners:
$ docker buildx create --use --name multiarch --driver docker-container
ERROR: could not create a builder instance with TLS data loaded from
environment. Please use `docker context create <context-name>` to create
a context for current environment and then create a builder instance
with context set to <context-name>
The dind service exports DOCKER_HOST=tcp://docker:2376 +
DOCKER_TLS_CERTDIR=/certs, but buildx --driver docker-container doesn't
read TLS from those env vars directly. Documented GitLab fix: create an
empty `docker context` (which inherits the current TLS env), then bind
buildx to that context name as a positional arg.
After this lands, the multi-arch buildx jobs should actually build and
push amd64 + arm64 images to
registry.gitlab.com/bigbodycobain/shadowbroker/backend:latest
registry.gitlab.com/bigbodycobain/shadowbroker/frontend:latest
Surfaced by the post-verification pipeline at
https://gitlab.com/bigbodycobain/Shadowbroker/-/pipelines/2550501798
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Pipelines on the GitLab mirror have been instant-failing with 0 jobs and
no started_at since the project was created — classic "shared runners
not allocated to unverified free-tier accounts" pattern. The account is
now identity-verified; this trivial comment bump exists solely to fire a
fresh pipeline that confirms runners now pick up the build-backend and
build-frontend jobs.
If the resulting pipeline produces real jobs that build the multi-arch
images and push them to registry.gitlab.com/bigbodycobain/shadowbroker/{backend,frontend},
the GitLab install path is at full parity with the GitHub one.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Re-cut v0.9.81 binaries from current main (which now includes the
private gate + DM hashchain spool from #326 and the gate-directory
test from #327). All three artifacts were signed with the same
minisign updater key as the original v0.9.81 release, so existing
v0.9.81 installs on Tauri auto-update accept the new bundles.
Updated hashes (verified against released assets):
- ShadowBroker_v0.9.81.zip f81f454bdc88e9a32c351df38212b8cfa624704d65764b971bb091eef62259c6
- ShadowBroker_0.9.81_x64-setup.exe 25e9a95d0d8ce959a7d08fe8e7406772ae24b596652793e81d1de5d02510a5a6
- ShadowBroker_0.9.81_x64_en-US.msi 34e655fc0c0f195ee4ac978f228a4b2b9d5565253b8771aca9ef4693409e9e70
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Adds the focused test Codex wrote alongside the gate-directory UI work
that already shipped in #326 (the `renderGateDirectory` helper used
both under the Infonet logo on the landing screen and as the output of
the `gates` command in the terminal).
The renderer itself is already on origin/main; this PR just ships the
test so CI catches regressions to the dual-variant render.
Verified locally:
- frontend npm run test:ci -- src/__tests__/mesh/infonetShellGateDirectory.test.tsx → 1/1 pass
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Private gate messages and offline DMs now ride the Infonet hashchain
as ciphertext-only events, replicated across nodes via private
transports (Tor onion / RNS / loopback) and decrypted only by parties
holding the gate or recipient keys.
Hashchain core (mesh_hashchain.py)
----------------------------------
* New ``append_private_gate_message`` and ``append_private_dm_message``
append paths with full signature verification, public-key binding,
revocation check, and replay protection in a dedicated sequence
domain (so a gate post does not consume the author's public broadcast
sequence, and a DM cannot replay-block a public message at sequence=1).
* Fork validation and full-chain validation now accept the gate
signature compatibility variants — older signatures that canonicalize
with/without epoch or reply_to still verify, so a re-sync from an
older peer doesn't reject still-valid history.
* DM hashchain spool: capped at 2 active sealed offline DMs per
recipient mailbox, plus a per-(sender, recipient) cap so one prolific
sender can't consume both slots. 1-hour TTL on the cap counter.
Spool intentionally small — it's an offline bootstrap channel,
not a persistent mailbox.
* Rebuild-state preserves the gate sequence domain across reloads so
a chain reload doesn't accidentally let an old gate sequence
replay-collide on next append.
Schema enforcement (mesh_schema.py)
-----------------------------------
* Private gate + DM payloads have closed allowlists of fields.
Plaintext keys (``message``, ``plaintext``, ``_local_plaintext``,
``_local_reply_to``) are explicit rejection-bait — they raise before
the event ever touches the chain.
* DM ciphertext + nonce must look like base64-ish sealed bytes;
obvious base64-encoded plaintext shapes are rejected.
* ``transport_lock`` required: DM hashchain spool requires
``private_strong``; gate accepts ``private``/``private_strong``/
``rns``/``onion``.
Defense-in-depth at the network layer (main.py + mesh_public.py)
----------------------------------------------------------------
* ``_infonet_sync_response_events`` now silently redacts private events
(gate_message + dm_message) unless the request looks like a loopback /
onion / RNS / private transport caller. If an operator accidentally
exposes :8000 to the public internet, an external puller gets
public events only — never ciphertext.
* ``_sync_from_peer`` raises ``PeerSyncRateLimited`` for 429 (handled
as 4-tuple return with retry_after_s) and ``PeerSyncHTTPError`` for
other non-200 statuses (handled by ``_run_public_sync_cycle`` to
honor server cooldown hints even outside the 429 path).
DM relay hydration (main.py)
-----------------------------
* New ``_hydrate_dm_relay_from_chain``: when accepted dm_message chain
events arrive on a node, they get deposited into the local DM relay
store with a deterministic sender_token_hash so re-sync of the same
event is idempotent. Recipients see the ciphertext as a normal DM
on their next poll and decrypt with their existing recipient key.
Other surfaces
--------------
* meshnode.bat / meshnode.sh now set ``MESH_INFONET_ALLOW_CLEARNET_SYNC=
false`` and the participant runtime flags by default so a freshly
spun-up node defaults to private-only sync.
* InfonetTerminal/InfonetShell.tsx adds a gate directory renderer for
the new private-gate workflow.
* docker-compose.relay.yml binds the relay backend to 127.0.0.1:8000
only; Tor's hidden service forwards onion traffic into 127.0.0.1.
Public clearnet :8000 stays off the network edge.
Tests
-----
* 7 new tests in test_private_gate_hashchain.py + test_private_dm_
hashchain.py covering: gate fork accepts ciphertext propagation,
gate fork rejects plaintext, append rejects plaintext before
normalize, append requires private_strong, append rejects
non-sealed ciphertext shape, DM spool 2-per-recipient + 1-per-pair
cap, DM hydration delivers to poll/claim.
* Updated test_mesh_node_bootstrap_runtime.py covers 429 backoff via
PeerSyncRateLimited 4-tuple AND PeerSyncHTTPError exception.
* Updated test_s14b_public_sync_gate_filter.py + test_s9b_gate_store_
hydration.py + test_gate_write_cutover.py cover the new private
redaction on public sync responses.
* test_private_gate_hashchain.py + test_private_dm_hashchain.py:
10 passed locally.
* Combined mesh-relevant suite (the 5 modified existing tests +
2 new): 17 passed.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* fix(start-scripts): find bundled privacy_core.dll next to script
start.bat and start.sh only checked the source-tree DLL path
(``privacy-core/target/release/privacy_core.dll``), not the bundled
location where MSI/AppImage/DMG installers stage the library directly
next to the script in backend-runtime/.
Users running start.bat from inside an MSI install dir (a documented
workaround when the desktop shell crashes) saw a scary "install Rust"
warning even though the DLL was sitting right next to them. See issue
#319 for the user-reported confusion.
Fix: add a fallback check for the bundled location before falling
through to the "build privacy-core from source" warning. Source-tree
behavior unchanged — the source path is still preferred when present.
Also re-stamps the v0.9.81 source archive: ``release_digests.json``
v0.9.81 zip hash updated to point at the rebuilt source archive that
contains these script changes. MSI/EXE/sig hashes are unchanged (the
scripts live at the repo root, not inside the desktop bundle).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(#319): bundle start.bat + start.sh into the MSI/EXE installers
Follow-up to the start-script DLL fallback fix in the prior commit.
ChrisMTheMan's report on #319 made it clear the workaround flow was:
1. MSI install crashes on launch (different bug, fixed in v0.9.81)
2. User goes looking for start.bat to launch the backend manually
3. start.bat isn't in their install dir, so they go fetch it from GitHub
4. They get a working script but it doesn't know about the bundled
privacy_core.dll layout, so they see a scary "install Rust" warning
The prior commit fixed step 4. This commit fixes step 3 — start.bat and
start.sh now ship inside the MSI/EXE installers (staged into
backend-runtime/ next to the privacy_core.dll they expect to find).
After the rebuild lands, an MSI user looking for these scripts finds
them right inside their install dir, already pointing at the correct
bundled DLL location.
What changed
------------
* ``build-backend-runtime.cjs`` now has a ``stageStartScripts()`` step
that copies start.bat and start.sh from the repo root into the
staged backend-runtime/. Preserves the executable bit on .sh under
POSIX.
* ``release_digests.json`` v0.9.81 block hashes refreshed for the
rebuilt MSI / EXE / source-zip (the scripts being bundled changed
the MSI/EXE contents; the source zip also includes the start-script
fix from the prior commit).
ShadowBroker_v0.9.81.zip 6.06 MB
af8c87ccdece8fbb9aadc6be63cce10d3fcba74e6d87ef83289dda6d555fd270
ShadowBroker_0.9.81_x64_en-US.msi 122.4 MB
8977c9a1c54e1f0d030436be9c4e3d81d766cc0080699eb747649095f360c7ff
ShadowBroker_0.9.81_x64-setup.exe 76.5 MB
4e866fa0423c0c2470ed32f4809167a7815dc23ee7762b69e95681c1f3a28250
Post-merge plan
---------------
Force-move the v0.9.81 tag to this commit and replace ALL release
assets on the GitHub release: zip, msi, exe, both .sig files,
latest.json, SHA256SUMS.txt, release-manifest.json.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
BigBodyCobain