Mesh Networking
The WeftOS mesh as of v0.8.4: transports (TCP, WebSocket, QUIC), Noise encryption, peer admission, the machine mesh service, wire frames, cross-node IPC, artifact and topic transfer, and the [kernel.mesh] configuration.
The mesh connects WeftOS nodes with encrypted peer-to-peer links. In the v0.8.4 daemon it carries cross-node IPC, liveness pings, artifact transfer for workload placement, inference proxying, assessment sync, and cogbus topics for exposed data. The original K6 design also described chain and tree synchronisation and process and service gossip over the mesh; parts of that are library code that the daemon does not run, and this page marks them.
Feature: mesh (default in the kernel build; QUIC is the quic feature, also default).
Enabled by: [kernel.mesh] enabled = true; the default is false.
Boot phase: 5d (after the A2A router, before cluster).
Source: crates/clawft-kernel/src/mesh*.rs; the configuration type is MeshConfig in crates/clawft-types/src/config/kernel.rs.
Related pages: Discovery (how peers find each other), Clustering (node state, liveness, and which replicated-state pieces run), Project topology (the machine mesh service and who owns the listener).
Layers
+-------------------------------------------------------------------+
| APPLICATION mesh_ipc (envelopes), mesh_artifact_*, mesh_swarm_*, |
| infer_proxy, mesh_topic_frames, mesh_assess, |
| mesh_liveness, workload_ctl |
+-------------------------------------------------------------------+
| ADMISSION mesh_admit, mesh_admit_gate (off | observe | enforce)|
+-------------------------------------------------------------------+
| FRAMING mesh_framing [4-byte length][1-byte type][payload] |
+-------------------------------------------------------------------+
| ENCRYPTION mesh_noise (Noise XX / IK, optional ML-KEM-768) |
+-------------------------------------------------------------------+
| TRANSPORT mesh (traits), mesh_tcp, mesh_ws, mesh_quic |
+-------------------------------------------------------------------+
| IDENTITY cluster (NodeIdentity), node_id, ipc (GlobalPid) |
+-------------------------------------------------------------------+Beside these are modules that implement designs from the original K6 plan but
have no caller in the daemon: mesh_dedup, mesh_listener, mesh_log,
mesh_bootstrap, mesh_mdns, mesh_kad (the discovery backends), and, on the
clustering side, the process table and the chain and
tree synchronisation helpers. Each page says which pieces are which.
Transports
The transport traits are MeshTransport (name, listen, connect,
supports) and MeshStream (send, recv, close, remote_addr)
(crates/clawft-kernel/src/mesh.rs). Three backends exist:
| Transport | transport = | Address | Use |
|---|---|---|---|
| TCP | tcp (the default) | host:port or tcp://... | development, networks that block UDP |
| WebSocket | ws or websocket | ws://..., wss://... | browsers, WASI, HTTP proxies |
| QUIC | quic | quic://host:port | multiplexed, UDP; built on quinn |
Over QUIC the TLS layer only completes the QUIC handshake: the listener mints an
ephemeral self-signed certificate and the client skips certificate verification.
Peer authentication and encryption are Noise, on top of the stream
(mesh_quic.rs, ADR-026). If transport = "quic" is set on a build without the
quic feature, boot logs an error and falls back to TCP
(docs/guides/mesh-quic.md).
Encryption
With noise = true, every peer connection runs a Noise handshake
(Noise_XX_25519_ChaChaPoly_SHA256). XX is first contact with mutual
authentication; IK is for a known responder key in one round trip
(NoisePattern in mesh_noise.rs). After the handshake, a hybrid upgrade can run
inside the encrypted channel: ML-KEM-768 key encapsulation, negotiated through
WeftHandshake.kem_supported. noise defaults to false; noise_key_path names
the key, and without it an ephemeral key is generated at boot.
Admission
Encryption says the channel is private; admission says who is on it.
After Noise, a peer sends an AdmitHello as its first frame. It signs the
session's handshake hash, so a captured hello cannot be replayed on another
session, and it binds the node id to the Ed25519 key the id derives from
(mesh_admit.rs). The node id is hex(SHA-256(pubkey)[..16])
(crates/clawft-kernel/src/node_id.rs).
admission in [kernel.mesh] selects the policy (ADR-103, P3-K1):
| Mode | Behaviour |
|---|---|
off | no policy, but a hello that arrives is still verified and its source_node bound to the key |
observe (default) | checks and records would-be refusals; never refuses and never marks a peer admitted. Takes effect once genesis_hash is pinned |
enforce | refuses unsigned, plaintext, wrong-genesis, revoked and verdict-denied peers. Needs genesis_hash and a governance gate, unless admission_open_membership is set |
For any peer that is not admitted, the listener strips the envelope's src_scope
before delivery. genesis_hash is a cluster label, not a credential: anyone who
knows it can present it. A peer's class (node or leaf) is self-declared in its
signed hello. Two per-listener limits apply under every mode:
max_connections_per_ip (default 64) and first_frame_timeout_secs (default 10).
Wire framing
mesh_framing.rs defines a typed frame, [4-byte big-endian length][1-byte type][payload], at most 16 MiB (MAX_MESSAGE_SIZE in mesh.rs), and a table of
frame types. Not every type is in use. The daemon sends cross-node IPC as a
MeshIpcEnvelope serialised to JSON and dispatches received bytes in order:
assessment sync, then any typed-frame handler registered for the leading type
byte, then the JSON envelope (MeshRuntime::handle_incoming_tallied,
mesh_runtime.rs). The last column records what I found by searching crates/
for non-test uses of each variant:
| Type | Value | Carries | Used by the daemon |
|---|---|---|---|
Handshake | 0x01 | WeftHandshake after Noise | yes (transports) |
IpcMessage | 0x02 | a KernelMessage envelope | no non-test use of the variant; envelopes are sent as JSON |
ChainSync | 0x03 | chain sync request and response | no non-test use |
TreeSync | 0x04 | Merkle tree sync | no non-test use |
ServiceAdvert | 0x05 | service advertisement | no non-test use |
ProcessAdvert | 0x06 | process advertisement | no non-test use |
Heartbeat | 0x07 | SWIM ping and pong | no non-test use; liveness uses the mesh.ping topic |
JoinRequest, JoinResponse | 0x08, 0x09 | cluster join | no; the join structs are never serialised into these frames (mesh_listener.rs notes) |
SyncDigest | 0x0A | compact state digest | no non-test use |
ArtifactRequest, ArtifactResponse | 0x0B, 0x0C | artifact control and piece data | yes (artifact transfer) |
LogAggregation | 0x0D | log aggregation | no non-test use |
AssessmentSync | 0x0E | cross-project assessment sync | yes |
SensorObservation | 0x0F | signed sensor observation payload | yes |
InferRequest, InferResponse | 0x10, 0x11 | one OpenAI-compatible request for an exposed inference role, and its response | yes |
TopicRequest, TopicData | 0x12, 0x13 | cogbus mesh topic subscribe, credit, data | yes |
The inference and topic frames are served only to admitted, verified peers.
WeftHandshake carries node_id, governance_genesis_hash,
governance_version, capabilities, kem_supported, chain_seq and
supported_sync_streams. Peers with a different governance genesis hash are
rejected.
Cross-node IPC
A message to another node travels in a MeshIpcEnvelope (mesh_ipc.rs):
source_node, dest_node, the KernelMessage, a hop_count (at most 8), a UUID
envelope_id, and optional src_scope and dest_scope. A peer's dest_scope is
a request that the kernel ignores.
Two supporting pieces exist as library code only: DedupFilter (a time-bounded
duplicate filter, 60 second TTL and 10,000 entries by default) is never called by
MeshRuntime::handle_incoming, and the service-resolution cache with 30 second
negative entries (ServiceResolutionCache) has no caller outside its module. The
WEFT-151 audit notes in mesh_dedup.rs record this and schedule wiring for 0.8.x.
Moving larger things
- Artifacts. Workload packages and model shards are content-addressed by
BLAKE3 and move in verified pieces with a
havebitfield, so a node that has finished a download can serve it (mesh_artifact_*,mesh_swarm_*, ADR-099 section 6). Only artifacts whose signed manifest verifies are seeded. - Inference. An exposed model role is proxied as
InferRequestandInferResponseframes (ADR-101 section 5). - Topics. Cogbus readings, stream chunks and
blob slices cross as
TopicData, against credit the subscriber grants.
Configuration
[kernel.mesh]
enabled = true
transport = "quic" # tcp (default) | ws | quic
listen_addr = "0.0.0.0:9489" # default 127.0.0.1:9489; `listen` is an alias
noise = true
seed_peers = ["quic://192.0.2.10:9489"]
admission = "observe" # off | observe (default) | enforceThe fields of MeshConfig (crates/clawft-types/src/config/kernel.rs):
| Field | Default | Notes |
|---|---|---|
enabled | false | when true, a failed bind aborts boot |
transport | "tcp" | tcp, ws/websocket, quic |
listen_addr | 127.0.0.1:9489 | loopback by default; peers on other machines need an explicit address. Port 9489 is "the weave" (ADR-103 D1) |
discovery | false | creates the Kademlia routing table and heartbeat tracker at boot |
seed_peers | [] | an entry may be address#node-id to pin the id the seed must claim |
noise, noise_key_path | false, none | see Encryption |
admission, genesis_hash, admission_open_membership | observe, none, false | see Admission |
mesh_nonce | none | 64 hex characters, identical on every node of the mesh; with the genesis pin it derives the mesh id a Seed binding is for (Seed licence proxy). Not a secret. Generate with weaver mesh nonce generate |
max_connections_per_ip, first_frame_timeout_secs | 64, 10 | apply under every admission mode |
service, service_socket | auto, none | relation to the machine mesh service, below |
leaf_listen_addr, leaf_listen_any, leaf_discovery_any | none, false, false | where the certified-leaf listeners bind, kept apart from listen_addr so a machine can accept leaves on its LAN while the mesh listener stays on loopback. An IP uses port main+2 for the leaf TCP listener and main+1 for leaf discovery. A list binds one pair per entry. A wildcard needs leaf_listen_any = true |
The machine mesh service
On a shared host, one machine mesh service owns the listener, Noise sessions and
admission, and each user daemon is a client over a local socket
(WEFTOS_MESH_SOCKET, default /var/run/weftos/mesh.sock). service is auto
(use the service when its socket answers and verifies, else run the mesh inside
the daemon), required (boot fails when no service answers) or off. A service
that answers but fails verification is a boot failure under every policy, never a
silent fallback (mesh_mode.rs). The roles and the commands are on
Project topology.
Operating the mesh
weaver mesh id # this node's MeshId and where it came from
weaver mesh create # mint a MeshId on a primary with no Seed licence
weaver mesh identity # node id, fingerprint, public key (never the private key)
weaver mesh pair list # pair requests for project work (ADR-108 P2b)
weaver cluster nodes --facts # nodes and their signed capability facts
weaver fleet status # every known node with trust, last seen, cogs, locationTest fixtures
clawft_kernel::mesh_test_support provides an in-memory mem:// transport, a
fault-injecting transport (drop, fail-next-connect, latency), a mock peer and a
mock clock. Time-dependent components (the heartbeat tracker, DedupFilter, the
service cache) take an injectable Clock, so tests are deterministic.
DEMOCRITUS Cognitive Loop
Continuous nervous system loop: Sense → Embed → Search → Update → Commit. Integrates ImpulseQueue, HNSW, CausalGraph, and CrossRefStore on every cognitive tick.
Discovery
How a WeftOS node finds peers at v0.8.4: seed peers dialed at boot, pairing, the leaf discovery socket, and the DiscoveryBackend library (bootstrap, peer exchange, mDNS, Kademlia) and which parts the daemon actually runs.