wireguard-mesh 0.2.5

Description:

wireguardmesh 0.2.5

Automatic L2 Wireguard Mesh Deployment
This is a small and opinionated tool that assists with the bring-up and
management of L2-bridged Wireguard mesh networks using only SSH.
It is intended to be used within IaC systems, CI pipelines, or other
automation sources able to provide state management & idempotent deployments.
Personally, I've developed this package for use with Pulumi and RKE to create my infrastructure network,
which spans across two LANs and multiple (Linode) cloud nodes.
I wanted to minimize configuration requirements to the bare minimum, so the only thing needed to get a
mesh going with this tool is a mesh.yaml file defining a minimal set of parameters for each node.
Otherwise, users are only expected to pre-configure SSH connectivity and any necessary NAT UDP ports mappings for Wireguard.
Features

Fully-connected topology or user-defined peering.
SSH-based peer configuration.

Automatic public, private and pre-shared key generation.
The configuring node only needs remote root/sudo access, and does not store private keys or any additional state.


Peerwise gretap/ip6gretap L2 links over Wireguard.
iproute2-based bridging with STP enabled and configurable priorities.

Non-Features
What this doesn't do:

Dynamically add/remove nodes or rotate keys without bringing down the mesh.
Configure IP forwarding, Internet routing, or DNS.
Support clients, gateways/egress/ingress, or anything other than a fully- or mostly-connected mesh of servers.

TODO

Add --dry-run flag to preview changes.
Add more accessibility to Wireguard config options.
Support some method of manual insecure interface bridging, e.g. for VPCs/VLANs.

How It Works
Because Wireguard doesn't support L2 or have the ability to route the same subnet across AllowedIPs for multiple peers,
creating a fully connected mesh at L2 requires a GRE tunnel to each peer, and a bridge for all peer tunnels on each node.
+-------------------------+ +-------------------------+
| node1: | | nodeN: |
| +-------------+ | | +-------------+ |
| | bridge | | | | bridge | |
| | +---------+ | +----+ | | +----+ | +---------+ | |
| | | gretap1 |<-->| | | | | |<-->| gretap1 | | |
| | +---------+ | | | | | | | | +---------+ | |
| | ... | | wg |<---->| wg | | ... | |
| | +---------+ | | | | | | | | +---------+ | |
| | | gretapN |<-->| | | | | |<-->| gretapN | | |
| | +---------+ | +----+ | | +----+ | +---------+ | |
| +-------------+ | | +-------------+ |
+-------------------------+ +-------------------------+
\ /
\--------- ssh ---------/
|
+----------------------+
| wireguard-mesh host |
+----------------------+

In this diagram, the bridge on each node has an assigned IP address within the defined network, and
STP prevents routing loops and advertisement floods that would otherwise be caused by multiple redundant links.
The Wireguard peer addresses are randomly chosen by default and only used for GRE tunneling.
Example mesh.yaml
Configuration files can also be provided from stdin, and JSON-formatting is available with the -j flag.
name: test
network: fd00:0:0:1::/64
full: false # Optional: default is true. Set to false for manual peering.
nodes:
mesh0:
addr: fd00:0:0:1::1/64
ssh: test0
endpoint: lan1.example.com
peers: # When unset or empty and full=false, all other nodes are peered.
- mesh1
- mesh2
- mesh3
mesh1:
addr: fd00:0:0:1::2/64
ssh: test1
endpoint: lan1.example.com:51821
mesh2:
addr: fd00:0:0:1::3/64
ssh: test2
endpoint: lan2.example.com
listen_port: 51850
mesh3:
addr: fd00:0:0:1::4/64
ssh: test3
endpoint: test3.cloud.example.com
prio: 100

Prior versions supported auto-assignment of addresses, but this has been removed in favor of explicit configuration.
Each node's addr field must be a unique interface address within the network.
Usage
Create a mesh.yaml as shown above, and then run:
mesh up

Show mesh info:
mesh info

Bring down the mesh and remove configs:
mesh down -r

Full Usage
usage: mesh [-h] [-j] [-J] [-f FILE] [-q] COMMAND ...

Wireguard mesh network manager.

options:
-h, --help show this help message and exit
-j, --json Input JSON instead of YAML (default: use file ext).
-J, --json-out Output JSON instead of YAML.
-f FILE, --file FILE Mesh configuration file or - for stdin (default: mesh.yaml).
-q, --quiet Suppress output.

commands:
COMMAND Command:
up - Bring up mesh.
down - Bring down mesh.
sync - Sync mesh.
show - Show mesh Wireguard info.
info - Show mesh network info.

-------------------------------
usage: mesh up [-h] [-i]

options:
-h, --help show this help message and exit
-i, --info Show mesh info after bringing up.

-------------------------------
usage: mesh down [-h] [-r]

options:
-h, --help show this help message and exit
-r, --remove Remove Wireguard interface configs.

System Requirements
Before deploying, the following is expected:

This package is installed on the configuring host.
Root or sudo access is available via SSH on all nodes.

Typically defined as entries in ~/.ssh/config.
Specified by a host/connection string or keyword argument dictionary for fabric.Connection.


Wireguard and wg-quick from wireguard-tools are installed on all nodes.
iproute2: ip6gretap and bridge capability
Optional: netcat and shell /dev/udp capability for reachability testing.

Network/Firewall:
This tool assumes that we have:

SSH accessibility from the configuring host to every node
Wireguard Endpoint UDP NAT ports mapped to LAN nodes

Notes
A static mesh is defined here as a configuration that does not change between creation and
deletion. This tool is capable of discerning the mesh state from the configuration
and has some ability to handle new peers, but deconfiguring/removing peers will leave dangling configuration files.
Hence, the recommended method to handle mesh changes is to perform a full replacement:
bring down the mesh from the existing configuration, and recreate from the new
configuration. This has the side effect of generating all new private and pre-shared keys as well.
Scalability
For a mesh of N fully-connected nodes, each node requires one Wireguard server with N-1 peers,
N-1 GRE tunnels, and one bridge interface for the tunnels.
Considerations:

STP will likely be very slow to learn/adapt with large N if links change state frequently.

Sometimes, STP doesn't find the best path between LAN-local nodes and ends up
adding unnecessary multi-hop latency, even with only 3 nodes.

The newly added prio node field allows setting of bridge priorities. Defaults are mapped to node indices, which lessens
the likelihood of arbitrarily bad routing decisions by ensuring that the 0-indexed node is the most likely root node (out of every 16).


There are better dynamic routing options, but STP is the simplest to enable.


Kernel limitations might prevent a large number of ip6gretap links?

Performance:

Links take a slight MTU hit for the GRE tunnels, on top of Wireguard's 80 bytes.
Latency differences in ping times were under 500 microseconds on average,
when comparing the same LAN vs meshed-over-LAN links.

LAN-local peers can sometimes find local endpoints for clients, even when the original endpoints
referred to the same NAT gateway.
STP can't provide any minimum latency or best-path guarantees.


iperf measurements TBD.

Security
Every Wireguard peering is automatically assigned a preshared key when the mesh is created, and each node's
private keys are automatically generated as well.
Keys are never saved anywhere other than each nodes /etc/wireguard/ config, and only loaded into memory when working
with meshes that were previously configured.
However, access to all keys is necessarily possible via SSH when configured for this tool,
so SSH identities should be properly protected within production environments.
Reference
Articles

Wireguard L2 gist
Comparison of Wireguard mesh tools

Tools & Libraries

wireguard-tools

Python library used here for configuration management.
Netlink-capable: Useful for development of active Wireguard management solutions.


wg-meshconf

(Python) CSV-based mesh management.