MLS export/reset and accept use live HTTP so uvicorn privacy-core state stays consistent; relay persistence and sender_seal fixes enable invite-accept-shared decrypt across onion peers. Adds participant/e2e compose overlays and harness recovery with optional Tor-only replicate mode.
Co-authored-by: Cursor <cursoragent@cursor.com>
Private gate messages and offline DMs now ride the Infonet hashchain
as ciphertext-only events, replicated across nodes via private
transports (Tor onion / RNS / loopback) and decrypted only by parties
holding the gate or recipient keys.
Hashchain core (mesh_hashchain.py)
----------------------------------
* New ``append_private_gate_message`` and ``append_private_dm_message``
append paths with full signature verification, public-key binding,
revocation check, and replay protection in a dedicated sequence
domain (so a gate post does not consume the author's public broadcast
sequence, and a DM cannot replay-block a public message at sequence=1).
* Fork validation and full-chain validation now accept the gate
signature compatibility variants — older signatures that canonicalize
with/without epoch or reply_to still verify, so a re-sync from an
older peer doesn't reject still-valid history.
* DM hashchain spool: capped at 2 active sealed offline DMs per
recipient mailbox, plus a per-(sender, recipient) cap so one prolific
sender can't consume both slots. 1-hour TTL on the cap counter.
Spool intentionally small — it's an offline bootstrap channel,
not a persistent mailbox.
* Rebuild-state preserves the gate sequence domain across reloads so
a chain reload doesn't accidentally let an old gate sequence
replay-collide on next append.
Schema enforcement (mesh_schema.py)
-----------------------------------
* Private gate + DM payloads have closed allowlists of fields.
Plaintext keys (``message``, ``plaintext``, ``_local_plaintext``,
``_local_reply_to``) are explicit rejection-bait — they raise before
the event ever touches the chain.
* DM ciphertext + nonce must look like base64-ish sealed bytes;
obvious base64-encoded plaintext shapes are rejected.
* ``transport_lock`` required: DM hashchain spool requires
``private_strong``; gate accepts ``private``/``private_strong``/
``rns``/``onion``.
Defense-in-depth at the network layer (main.py + mesh_public.py)
----------------------------------------------------------------
* ``_infonet_sync_response_events`` now silently redacts private events
(gate_message + dm_message) unless the request looks like a loopback /
onion / RNS / private transport caller. If an operator accidentally
exposes :8000 to the public internet, an external puller gets
public events only — never ciphertext.
* ``_sync_from_peer`` raises ``PeerSyncRateLimited`` for 429 (handled
as 4-tuple return with retry_after_s) and ``PeerSyncHTTPError`` for
other non-200 statuses (handled by ``_run_public_sync_cycle`` to
honor server cooldown hints even outside the 429 path).
DM relay hydration (main.py)
-----------------------------
* New ``_hydrate_dm_relay_from_chain``: when accepted dm_message chain
events arrive on a node, they get deposited into the local DM relay
store with a deterministic sender_token_hash so re-sync of the same
event is idempotent. Recipients see the ciphertext as a normal DM
on their next poll and decrypt with their existing recipient key.
Other surfaces
--------------
* meshnode.bat / meshnode.sh now set ``MESH_INFONET_ALLOW_CLEARNET_SYNC=
false`` and the participant runtime flags by default so a freshly
spun-up node defaults to private-only sync.
* InfonetTerminal/InfonetShell.tsx adds a gate directory renderer for
the new private-gate workflow.
* docker-compose.relay.yml binds the relay backend to 127.0.0.1:8000
only; Tor's hidden service forwards onion traffic into 127.0.0.1.
Public clearnet :8000 stays off the network edge.
Tests
-----
* 7 new tests in test_private_gate_hashchain.py + test_private_dm_
hashchain.py covering: gate fork accepts ciphertext propagation,
gate fork rejects plaintext, append rejects plaintext before
normalize, append requires private_strong, append rejects
non-sealed ciphertext shape, DM spool 2-per-recipient + 1-per-pair
cap, DM hydration delivers to poll/claim.
* Updated test_mesh_node_bootstrap_runtime.py covers 429 backoff via
PeerSyncRateLimited 4-tuple AND PeerSyncHTTPError exception.
* Updated test_s14b_public_sync_gate_filter.py + test_s9b_gate_store_
hydration.py + test_gate_write_cutover.py cover the new private
redaction on public sync responses.
* test_private_gate_hashchain.py + test_private_dm_hashchain.py:
10 passed locally.
* Combined mesh-relevant suite (the 5 modified existing tests +
2 new): 17 passed.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Second commit on this branch (first added the per-sender cap + accept_replica
primitive). This commit wires the actual cross-node propagation:
Outbound (sender side)
----------------------
* New ``DMRelay._replicate_envelope_to_peers_async()`` — fire-and-forget
thread that POSTs the envelope to every authenticated relay peer via
the same per-peer HMAC pattern gate-message replication uses (#256
``X-Peer-Url`` + ``X-Peer-HMAC`` headers, ``resolve_peer_key_for_url``).
* ``deposit()`` now calls the replication helper after a successful
local accept. Per-peer errors are swallowed — slow Tor peers must not
block the sender's UX, and the recipient polling from a healthy peer
works fine even if some peers are down.
* Metrics: dm_replication_push_ok / _rejected / _error.
Inbound (receiving side)
------------------------
* New endpoint ``POST /api/mesh/dm/replicate-envelope`` in
routers/mesh_peer_sync.py.
* Same HMAC auth gate (``_verify_peer_push_hmac``) as the existing
infonet/gate peer-push endpoints. Unauthenticated requests get 403.
* Body cap of 64 KB (DM envelope is bounded by MESH_DM_MAX_MSG_BYTES).
* Calls DMRelay.accept_replica which enforces the per-sender cap as a
network rule — hostile sender's relay can hold extras locally but
honest peers reject them on inbound replication.
End-to-end flow now works
-------------------------
1. Alice's node accepts a deposit to Bob's mailbox (local cap check).
2. Alice's node spawns a background thread that POSTs the envelope
to MESH_RELAY_PEERS with per-peer HMAC.
3. Each peer's /api/mesh/dm/replicate-envelope verifies the HMAC and
calls accept_replica, which re-enforces the per-sender cap.
4. Bob (offline at the time of send) eventually logs into ANY node
in MESH_RELAY_PEERS, his existing pollDmMailboxes pulls from
the local mailbox there, finds Alice's envelope, decrypts.
Tests
-----
backend/tests/test_dm_replicate_envelope_endpoint.py — 4 tests:
TestReplicateEndpointAuth:
- rejects requests without peer HMAC (403)
- rejects requests with WRONG peer HMAC (403) — confirms the
HMAC is actually verified, not just present
- rejects oversize bodies (>64 KB) with 400/413
TestReplicateEndpointRegistered:
- static check that POST /api/mesh/dm/replicate-envelope is
registered on app.routes — catches future refactor that
drops the router include
All 38 backend tests touching the new code paths still pass:
test_dm_relay_per_sender_cap.py (14)
test_dm_replicate_envelope_endpoint.py (4)
test_no_new_duplicate_routes.py (1) — new route is unique
test_per_peer_secret_resolver.py (19) — HMAC primitive unaffected
What's still ahead (PR-3+)
--------------------------
* ack propagation: when recipient pulls a message on node X, peers Y/Z
should prune their copies to free the sender's quota network-wide.
Without this, the sender's quota frees only on the node the recipient
actually polled — other peers still see N pending until TTL expiry.
Workable but suboptimal. PR-3 will add a /api/mesh/dm/ack endpoint
with the same HMAC pattern.
* recipient pull-from-peers: today the recipient's poll only hits
their own node's relay. If they log into a peer they didn't deposit
with, they need a way to fetch envelopes from other peers in
MESH_RELAY_PEERS. Today this works as long as the recipient's
current node is one of the peers Alice's node pushed to — which is
true in a fully-meshed deployment but not guaranteed for partial
meshes. PR-4 if telemetry shows this matters.
BigBodyCobain