Switch

Security warning: The Switch authenticates peers with a pre-shared key but does not encrypt traffic, and it deserializes Python pickle from every authenticated peer. Bound to anything other than loopback it requires EDRI_SWITCH_SECRET (or EDRI_SWITCH_SECRET_FILE); without one it starts on loopback only. Keep the port (default 8899) off the public internet and run it on a trusted, isolated network. See Security below.

The Switch is a standalone TCP/IP server that acts as a central hub connecting multiple EDRI Router across a network. While the Router is the local dispatcher inside a single EDRI instance, the Switch enables distributed event routing across machines, containers, or data centers.

        graph TB
    S((Switch))
    subgraph "Node 1"
        R1[Router A] --- C1[Connector A]
        C1 -- TCP/IP --> S
    end
    subgraph "Node 2"
        R2[Router B] --- C2[Connector B]
        C2 -- TCP/IP --> S
    end
    subgraph "Node 3"
        R3[Router C] --- C3[Connector C]
        C3 -- TCP/IP --> S
    end
    

The Switch itself is not a Router — it does not interpret event contents or maintain subscriptions. Payloads are opaque to it: it authenticates and re-frames each one but never deserializes it.

---

Architecture

The Switch is composed of three internal components that run as daemon threads:

Sender (per connection)

• Reads payloads from a per-connection queue.Queue.

• Wraps each one in an authenticated EVENT frame (see Message protocol) and sends it over the TCP socket. Everything the Switch sends downstream is an EVENT frame — the Connector dispatches on the unpickled Python type, so demands and last-events blobs need no distinct type in this direction.

• If the send fails (remote side disconnected), the payload is placed back on the queue and the thread exits.

• The queue never loses messages — they are re-queued on send failure.

Receiver (per connection)

• Reads authenticated frames from the TCP socket, verifying each frame's MAC before acting on it.

• Branches on the frame's type field:

  • DEMANDS and LAST_EVENTS — the payload is forwarded to all other connected routers.

  • EVENT — the 16-byte target field is the destination router UUID; the payload is forwarded only to that router.

• A frame that fails MAC verification, carries an out-of-order sequence number, or has an unexpected type drops the connection.

• On connection reset or socket closure, the thread exits gracefully.

Forwarder (single thread)

• Reads (router_id, forward_type, payload) tuples from a central queue.

  • ForwardType.SPECIFIC — forward the payload only to router_id.

  • ForwardType.OTHERS — forward the payload to every connected router except router_id.

• This decouples network I/O from routing logic.

A separate daemon thread serves the read-only status endpoint; it touches none of the forwarding path.

---

Message protocol

Every frame in either direction is one 48-byte header followed by a payload:

Offset	Size	Field	Description
0	1	version	Protocol version (1)
1	1	type	Frame type (see below)
2	2	flags	Reserved
4	8	seq	Per-direction sequence number, gap-free, never reset
12	4	length	Payload length in bytes (big-endian unsigned)
16	16	target	Destination router UUID on a Connector→Switch EVENT; zero otherwise
32	16	mac	HMAC-SHA256 (truncated to 16 bytes) over header[0:32] ‖ payload
48	…	payload	Serialized event or demands data (raw bytes)

The MAC covers every header byte except the MAC field itself, so type, sequence and target are authenticated, not just the body. It is keyed with the session key for that direction, derived during the handshake — not the pre-shared key itself. A frame whose MAC does not verify, or whose seq is not the next expected value, is rejected and the connection is dropped.

The payload cap is EDRI_SWITCH_MAX_MESSAGE (default 64 MiB); the length field is validated against it before the payload is read. Bulk data belongs on a SharedMemoryPipe, not the Switch.

The source router is not carried in each frame — it is the identity of the connection itself, established during the handshake. When the Switch forwards a payload to a destination router it builds a fresh EVENT frame with that connection's own keys and sequence numbers; frames are re-framed hop by hop, never passed through byte for byte.

Frame types

Type	Direction	Meaning
HELLO_SERVER	Switch → Connector	First handshake frame; carries the server nonce
HELLO_CLIENT	Connector → Switch	Router UUID, client nonce, and the client's transcript MAC
HELLO_ACCEPT	Switch → Connector	The server's transcript MAC; completes the handshake
EVENT	both	A shareable event; target names the destination router
DEMANDS	Connector → Switch	Router announces its event-type subscriptions
LAST_EVENTS	Connector → Switch	Router requests replay of events queued during an outage
ERROR	both	Reserved for a structured rejection reason

The types are defined in edri.protocol.frame.FrameType.

---

Connection lifecycle

1. Handshake

The Connector opens a TCP connection and runs a three-frame mutual handshake:

1. Switch → Connector: HELLO_SERVER with a random server nonce.

2. Connector → Switch: HELLO_CLIENT with the Router UUID, a client nonce, and a MAC over the transcript keyed by the pre-shared key.

3. Switch → Connector: HELLO_ACCEPT with the server's own transcript MAC.

Both sides prove possession of the pre-shared key, and the Router UUID is inside the MACed transcript, so an outsider cannot claim another router's identity. Each side then derives a pair of per-direction session keys from the two nonces and the pre-shared key; every frame after the handshake is MACed with those, never with the pre-shared key directly. A peer that fails the handshake has its connection dropped without a reply.

If a router with the same UUID is already connected, the newcomer — having proved key possession — evicts the incumbent: the old channel is closed and its Sender is stopped. A half-open socket left by a crashed node cannot be told from a healthy one, so this is what lets a restarted node reclaim its own identity. The new connection always starts with a fresh, empty queue.

2. Demand propagation

Immediately after the handshake, the Switch sends the demands (subscriptions) of every other connected router to the new router. This ensures the new router knows which event types other routers are interested in.

3. Ongoing communication

• Routers send shareable events to the Switch as EVENT frames addressed to specific remote routers.

• Routers broadcast their demands to all other routers on every change.

• The payload stays opaque to the Switch; it re-frames each one under the destination connection's keys and forwards it unread.

4. Disconnection and reconnection

When a TCP connection drops, the Switch keeps the Connection entry with its now-dead channel. Frames addressed to that router are still queued, but its Sender exits the first time it tries the dead socket, so the backlog is never delivered. When the router reconnects it gets a fresh connection with an empty queue, and that backlog is discarded.

The Switch therefore preserves nothing across a reconnect. Recovery is entirely the Connector's pull-based cache replay: on reconnect each side asks its peer for the ID of the last event it received and replays from the Router's in-memory cache everything after it, bounded by EDRI_CACHE_TIMEOUT. See Connector for that exchange.

---

Running the Switch

Standalone (Python)

python -m edri.switch.switch

This starts the Switch on localhost:8899 by default, with no authentication — fine for local development. To accept connections from other hosts, set EDRI_SWITCH_HOST (e.g. 0.0.0.0) and EDRI_SWITCH_SECRET; the Switch refuses to start on a non-loopback address without a secret.

Docker

The project publishes a versioned Switch image for every release tag. Use the version matching the EDRI package:

docker run -d --name edri-switch \
  -p 8899:8899 \
  -e EDRI_SWITCH_HOST=0.0.0.0 \
  -e EDRI_SWITCH_SECRET=<32+ byte key> \
  blastycz/edri:switch-<VERSION>

Prefer EDRI_SWITCH_SECRET_FILE with a mounted Docker or Kubernetes secret over an inline -e value, which is visible in docker inspect and the pod spec.

For the strongest reproducibility guarantee, pin the versioned image by its published SHA256 digest. The switch-latest tag points to the newest stable release; release candidates are available only through their explicit switch-<VERSION> tags. Avoid mutable tags in production.

Security: The -p 8899:8899 flag publishes the Switch port on the host's network. In a production environment, use -p 127.0.0.1:8899:8899 to bind to loopback only, or omit the -p flag entirely and rely on Docker's internal networking (--network host or a custom bridge network). The Switch port must never be exposed to the public internet. See Security below.

Docker (build locally)

docker build -f edri/switch/Dockerfile -t edri-switch .
docker run -d --name edri-switch -p 8899:8899 edri-switch

Configuration

Environment Variable	Default	Description
EDRI_SWITCH_HOST	localhost	Address to bind the Switch to
EDRI_SWITCH_PORT	8899	Port to bind the Switch to
EDRI_SWITCH_SECRET	(unset)	Pre-shared key the Switch and every Connector authenticate with; required for any non-loopback bind
EDRI_SWITCH_SECRET_FILE	(unset)	Path to a file holding one key per line (for key rotation); takes precedence over EDRI_SWITCH_SECRET
EDRI_SWITCH_MAX_MESSAGE	67108864	Largest accepted frame payload, in bytes (64 MiB)
EDRI_SWITCH_STATUS_ENABLED	true	Start the HTTP status endpoint (below)
EDRI_SWITCH_STATUS_HOST	localhost	Address to bind the status endpoint to
EDRI_SWITCH_STATUS_PORT	8898	Port for the status endpoint

A secret must be at least 32 bytes. Generate one with python -c "import secrets; print(secrets.token_urlsafe(32))".

The Switch logs at DEBUG level by default.

---

Status endpoint

The Switch lives outside EDRI's router/manager/event topology — it forwards raw bytes and never speaks the Event protocol — so the API Status endpoints machinery cannot be reused here. Instead it exposes the same kind of operational snapshot through a thin, read-only stdlib HTTP server running in a daemon thread (zero extra dependencies):

• GET /status (also served at /) → JSON. Any other path returns 404.

It binds to EDRI_SWITCH_STATUS_HOST:EDRI_SWITCH_STATUS_PORT (default localhost:8898) and can be turned off with EDRI_SWITCH_STATUS_ENABLED=false. A taken status port is logged and skipped — it never stops the Switch from forwarding. The handler only reads live structures, so it needs no locking and cannot perturb routing; a momentarily stale count is the only cost.

curl http://localhost:8898/status

{
  "connections": [
    {
      "router_id": "577e4ac2-6753-48b1-ac08-62097e5c5d7b",
      "peer": "127.0.0.1:35964",
      "queue_depth": 0,
      "has_demands": true,
      "demands_bytes": 142
    }
  ],
  "connections_total": 1,
  "forwarder_queue_depth": 0,
  "vitals": { "rss_bytes": 25477120, "cpu_seconds": 0.06, "threads": 4,
              "open_fds": 7, "pid": 2845344, "uptime_seconds": 1.94 }
}

Field	Meaning
connections[].router_id	UUID of the connected Router.
connections[].peer	Remote host:port of the TCP connection; null once the socket is closed.
connections[].queue_depth	Messages queued for that Router — its outbound backpressure (a Router that stops draining).
connections[].has_demands / demands_bytes	Whether the Router has announced subscriptions yet, and their size.
connections_total	Number of connected Routers.
forwarder_queue_depth	Depth of the single central Forwarder queue — the Switch's overall backpressure signal.
vitals	Switch process metrics (rss_bytes, rss_peak_bytes, cpu_seconds, threads, open_fds, pid, uptime_seconds); best-effort, null when unreadable on the platform.

---

Resilience guarantees

Scenario	Behavior
Router disconnects	Frames addressed to it are queued but never delivered; that connection's Sender exits once it hits the dead socket
Router reconnects (same ID)	The old connection is evicted and a fresh, empty queue is created; the accumulated backlog is discarded
Switch crashes	All in-memory queues are lost (no persistence)
Network partition	Events sent during the partition are lost by the Switch; recovery depends on the Connector's cache replay

Important: The Switch is a single-instance, in-memory component with no persistence and no built-in HA/failover, and it preserves nothing across a reconnect. The Router cache (EDRI_CACHE_TIMEOUT) is the only safety net for a Switch or link outage — the Connector replays missed events from it on reconnect (see Connector). For production resilience, monitor the Switch process and restart it promptly on failure.

---

Security

The Switch authenticates its peers, then forwards pickle-serialized payloads between them without inspecting them:

• Pre-shared key authentication — EDRI_SWITCH_SECRET (or EDRI_SWITCH_SECRET_FILE) is a cluster-wide key. The mutual handshake stops any peer without it from connecting or from impersonating another router. It does not authenticate members against each other: every node holds the same key. A Switch bound to any non-loopback address refuses to start without a secret; on loopback it starts without one and logs a warning. Generate one with python -c "import secrets; print(secrets.token_urlsafe(32))".

• No transport encryption — frame payloads travel over TCP in the clear. The MAC authenticates each frame but does not hide it; a network-level attacker can still read every event payload. Use a VPN, WireGuard tunnel, mTLS proxy, or service mesh for sensitive payloads.

• No payload validation — the Switch does not deserialize or validate what it forwards. An authenticated but compromised peer can still send arbitrary pickle to another peer, which unpickles it — and unpickling is code execution, so one compromised key holder runs code on every other node. Connect only nodes you trust as much as each other, restrict who holds the key, and rotate it if it may have leaked (see EDRI_SWITCH_SECRET_FILE in Environment Variables, which keeps the old and new key valid at once).

• Status endpoint — the HTTP status endpoint (default port 8898) has no authentication of its own and exposes operational metadata (router IDs, peer addresses, queue depths). It refuses to start off-loopback unless a Switch secret is set, and even then it stays readable by anyone who can reach the port — restrict access to monitoring systems.

Network policy (Kubernetes)

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: edri-switch-ingress
spec:
  podSelector:
    matchLabels:
      app: edri-switch
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: edri-app
      ports:
        - port: 8899

---

Related

• Connector — how a Router connects to the Switch

• Router — local event dispatching, subscriptions, and shareable events

• Event — event model, including shareable flag

• Environment Variables — environment variables for Switch configuration