Files
Shadowbroker/backend/tests/mesh/test_private_dm_hashchain.py
T
Shadowbroker 1d7fa5185a feat(infonet): private gate + DM hashchain spool with hardened propagation (#326)
Private gate messages and offline DMs now ride the Infonet hashchain
as ciphertext-only events, replicated across nodes via private
transports (Tor onion / RNS / loopback) and decrypted only by parties
holding the gate or recipient keys.

Hashchain core (mesh_hashchain.py)
----------------------------------

* New ``append_private_gate_message`` and ``append_private_dm_message``
  append paths with full signature verification, public-key binding,
  revocation check, and replay protection in a dedicated sequence
  domain (so a gate post does not consume the author's public broadcast
  sequence, and a DM cannot replay-block a public message at sequence=1).
* Fork validation and full-chain validation now accept the gate
  signature compatibility variants — older signatures that canonicalize
  with/without epoch or reply_to still verify, so a re-sync from an
  older peer doesn't reject still-valid history.
* DM hashchain spool: capped at 2 active sealed offline DMs per
  recipient mailbox, plus a per-(sender, recipient) cap so one prolific
  sender can't consume both slots. 1-hour TTL on the cap counter.
  Spool intentionally small — it's an offline bootstrap channel,
  not a persistent mailbox.
* Rebuild-state preserves the gate sequence domain across reloads so
  a chain reload doesn't accidentally let an old gate sequence
  replay-collide on next append.

Schema enforcement (mesh_schema.py)
-----------------------------------

* Private gate + DM payloads have closed allowlists of fields.
  Plaintext keys (``message``, ``plaintext``, ``_local_plaintext``,
  ``_local_reply_to``) are explicit rejection-bait — they raise before
  the event ever touches the chain.
* DM ciphertext + nonce must look like base64-ish sealed bytes;
  obvious base64-encoded plaintext shapes are rejected.
* ``transport_lock`` required: DM hashchain spool requires
  ``private_strong``; gate accepts ``private``/``private_strong``/
  ``rns``/``onion``.

Defense-in-depth at the network layer (main.py + mesh_public.py)
----------------------------------------------------------------

* ``_infonet_sync_response_events`` now silently redacts private events
  (gate_message + dm_message) unless the request looks like a loopback /
  onion / RNS / private transport caller. If an operator accidentally
  exposes :8000 to the public internet, an external puller gets
  public events only — never ciphertext.
* ``_sync_from_peer`` raises ``PeerSyncRateLimited`` for 429 (handled
  as 4-tuple return with retry_after_s) and ``PeerSyncHTTPError`` for
  other non-200 statuses (handled by ``_run_public_sync_cycle`` to
  honor server cooldown hints even outside the 429 path).

DM relay hydration (main.py)
-----------------------------

* New ``_hydrate_dm_relay_from_chain``: when accepted dm_message chain
  events arrive on a node, they get deposited into the local DM relay
  store with a deterministic sender_token_hash so re-sync of the same
  event is idempotent. Recipients see the ciphertext as a normal DM
  on their next poll and decrypt with their existing recipient key.

Other surfaces
--------------

* meshnode.bat / meshnode.sh now set ``MESH_INFONET_ALLOW_CLEARNET_SYNC=
  false`` and the participant runtime flags by default so a freshly
  spun-up node defaults to private-only sync.
* InfonetTerminal/InfonetShell.tsx adds a gate directory renderer for
  the new private-gate workflow.
* docker-compose.relay.yml binds the relay backend to 127.0.0.1:8000
  only; Tor's hidden service forwards onion traffic into 127.0.0.1.
  Public clearnet :8000 stays off the network edge.

Tests
-----

* 7 new tests in test_private_gate_hashchain.py + test_private_dm_
  hashchain.py covering: gate fork accepts ciphertext propagation,
  gate fork rejects plaintext, append rejects plaintext before
  normalize, append requires private_strong, append rejects
  non-sealed ciphertext shape, DM spool 2-per-recipient + 1-per-pair
  cap, DM hydration delivers to poll/claim.
* Updated test_mesh_node_bootstrap_runtime.py covers 429 backoff via
  PeerSyncRateLimited 4-tuple AND PeerSyncHTTPError exception.
* Updated test_s14b_public_sync_gate_filter.py + test_s9b_gate_store_
  hydration.py + test_gate_write_cutover.py cover the new private
  redaction on public sync responses.
* test_private_gate_hashchain.py + test_private_dm_hashchain.py:
  10 passed locally.
* Combined mesh-relevant suite (the 5 modified existing tests +
  2 new): 17 passed.

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-24 21:25:18 -06:00

214 lines
7.6 KiB
Python

import base64
import time
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric import ed25519
from services.config import get_settings
from services.mesh import mesh_crypto, mesh_dm_relay, mesh_hashchain, mesh_protocol, mesh_secure_storage
def _keypair():
private_key = ed25519.Ed25519PrivateKey.generate()
public_raw = private_key.public_key().public_bytes(
encoding=serialization.Encoding.Raw,
format=serialization.PublicFormat.Raw,
)
public_key = base64.b64encode(public_raw).decode("utf-8")
node_id = mesh_crypto.derive_node_id(public_key)
return private_key, public_key, node_id
def _payload(recipient_id: str = "recipient-a", msg_id: str = "dm-1") -> dict:
return mesh_protocol.normalize_payload(
"dm_message",
{
"recipient_id": recipient_id,
"delivery_class": "request",
"recipient_token": "",
"ciphertext": base64.b64encode(f"cipher-{msg_id}".encode("utf-8")).decode("ascii"),
"msg_id": msg_id,
"timestamp": int(time.time()),
"format": "mls1",
"transport_lock": "private_strong",
},
)
def _signature(private_key, node_id: str, sequence: int, payload: dict) -> str:
signature_payload = mesh_crypto.build_signature_payload(
event_type="dm_message",
node_id=node_id,
sequence=sequence,
payload=payload,
)
return private_key.sign(signature_payload.encode("utf-8")).hex()
def _fresh_infonet(tmp_path, monkeypatch) -> mesh_hashchain.Infonet:
monkeypatch.setattr(mesh_hashchain, "DATA_DIR", tmp_path)
monkeypatch.setattr(mesh_hashchain, "CHAIN_FILE", tmp_path / "infonet.json")
monkeypatch.setattr(mesh_hashchain, "WAL_FILE", tmp_path / "infonet.wal")
return mesh_hashchain.Infonet()
def _fresh_relay(tmp_path, monkeypatch) -> mesh_dm_relay.DMRelay:
monkeypatch.setattr(mesh_dm_relay, "DATA_DIR", tmp_path)
monkeypatch.setattr(mesh_dm_relay, "RELAY_FILE", tmp_path / "dm_relay.json")
monkeypatch.setattr(mesh_secure_storage, "DATA_DIR", tmp_path)
monkeypatch.setattr(mesh_secure_storage, "MASTER_KEY_FILE", tmp_path / "wormhole_secure_store.key")
get_settings.cache_clear()
return mesh_dm_relay.DMRelay()
def test_private_dm_hashchain_spools_two_ciphertexts_per_recipient_from_distinct_senders(tmp_path, monkeypatch):
inf = _fresh_infonet(tmp_path, monkeypatch)
senders = [_keypair(), _keypair()]
for idx, (private_key, public_key, node_id) in enumerate(senders, start=1):
payload = _payload(msg_id=f"dm-{idx}")
event = inf.append_private_dm_message(
node_id=node_id,
payload=payload,
signature=_signature(private_key, node_id, 1, payload),
sequence=1,
public_key=public_key,
public_key_algo="Ed25519",
protocol_version=mesh_protocol.PROTOCOL_VERSION,
timestamp=float(payload["timestamp"]),
)
assert event["event_type"] == "dm_message"
private_key, public_key, node_id = _keypair()
third = _payload(msg_id="dm-3")
try:
inf.append_private_dm_message(
node_id=node_id,
payload=third,
signature=_signature(private_key, node_id, 1, third),
sequence=1,
public_key=public_key,
public_key_algo="Ed25519",
protocol_version=mesh_protocol.PROTOCOL_VERSION,
timestamp=float(third["timestamp"]),
)
except ValueError as exc:
assert "spool full" in str(exc)
else:
raise AssertionError("third DM spool event was accepted")
for _private_key, _public_key, sender_node_id in senders:
assert inf.sequence_domains[f"{sender_node_id}|dm_message"] == 1
assert inf.validate_chain(verify_signatures=True)[0] is True
def test_private_dm_hashchain_limits_one_active_spool_per_sender_recipient_pair(tmp_path, monkeypatch):
inf = _fresh_infonet(tmp_path, monkeypatch)
private_key, public_key, node_id = _keypair()
first = _payload(msg_id="dm-1")
inf.append_private_dm_message(
node_id=node_id,
payload=first,
signature=_signature(private_key, node_id, 1, first),
sequence=1,
public_key=public_key,
public_key_algo="Ed25519",
protocol_version=mesh_protocol.PROTOCOL_VERSION,
timestamp=float(first["timestamp"]),
)
second = _payload(msg_id="dm-2")
try:
inf.append_private_dm_message(
node_id=node_id,
payload=second,
signature=_signature(private_key, node_id, 2, second),
sequence=2,
public_key=public_key,
public_key_algo="Ed25519",
protocol_version=mesh_protocol.PROTOCOL_VERSION,
timestamp=float(second["timestamp"]),
)
except ValueError as exc:
assert "sender spool full" in str(exc)
else:
raise AssertionError("second DM from same sender to same recipient was accepted")
def test_private_dm_hashchain_rejects_plaintext(tmp_path, monkeypatch):
inf = _fresh_infonet(tmp_path, monkeypatch)
private_key, public_key, node_id = _keypair()
payload = _payload()
payload["message"] = "plaintext"
try:
inf.append_private_dm_message(
node_id=node_id,
payload=payload,
signature=_signature(private_key, node_id, 1, _payload()),
sequence=1,
public_key=public_key,
public_key_algo="Ed25519",
protocol_version=mesh_protocol.PROTOCOL_VERSION,
)
except ValueError as exc:
assert "plaintext" in str(exc)
else:
raise AssertionError("private DM append accepted plaintext")
def test_private_dm_hashchain_rejects_non_sealed_ciphertext_shape(tmp_path, monkeypatch):
inf = _fresh_infonet(tmp_path, monkeypatch)
private_key, public_key, node_id = _keypair()
payload = _payload()
payload["ciphertext"] = "not sealed plaintext"
try:
inf.append_private_dm_message(
node_id=node_id,
payload=payload,
signature=_signature(private_key, node_id, 1, payload),
sequence=1,
public_key=public_key,
public_key_algo="Ed25519",
protocol_version=mesh_protocol.PROTOCOL_VERSION,
)
except ValueError as exc:
assert "sealed bytes" in str(exc)
else:
raise AssertionError("private DM append accepted non-base64 ciphertext")
def test_hydrate_dm_relay_from_chain_delivers_to_poll_claim(tmp_path, monkeypatch):
inf = _fresh_infonet(tmp_path / "chain", monkeypatch)
relay = _fresh_relay(tmp_path / "relay", monkeypatch)
monkeypatch.setattr(mesh_hashchain, "infonet", inf)
monkeypatch.setattr(mesh_dm_relay, "dm_relay", relay)
private_key, public_key, node_id = _keypair()
payload = _payload(recipient_id="recipient-a", msg_id="dm-chain-1")
event = inf.append_private_dm_message(
node_id=node_id,
payload=payload,
signature=_signature(private_key, node_id, 1, payload),
sequence=1,
public_key=public_key,
public_key_algo="Ed25519",
protocol_version=mesh_protocol.PROTOCOL_VERSION,
timestamp=float(payload["timestamp"]),
)
from main import _hydrate_dm_relay_from_chain
assert _hydrate_dm_relay_from_chain([event]) == 1
messages, more = relay.collect_claims(
"recipient-a",
[{"type": "requests", "token": "recipient-request-token"}],
limit=8,
)
assert more is False
assert [message["msg_id"] for message in messages] == ["dm-chain-1"]
assert messages[0]["ciphertext"] == payload["ciphertext"]