WireGuard on Kubernetes

Road-warrior-style VPN server

(See comments on Hacker News.)

WireGuard was first introduced in Linux kernel 5.6, but Ubuntu 20.04 LTS includes a backport in its 5.4 kernel.

This means that if your Kubernetes nodes are running Ubuntu 20.04 LTS or later, they come with WireGuard installed as a kernel module that will automatically load when needed. If your cluster permits to you to set CAP_NET_ADMIN on containers in pods, you can run a road-warrior-style WireGuard server in Kubernetes without modifying the node.

Example deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: wireguard
  namespace: myvpn
  labels:
    app: wireguard
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: wireguard
  template:
    metadata:
      labels:
        app: wireguard
    spec:
      restartPolicy: Always
      volumes:
        - name: wg0-key
          secret:
            secretName: wg0-key
        - name: wg0-conf
          configMap:
            name: wg0-conf
      containers:
        - name: wireguard
          image: sclevine/wg
          imagePullPolicy: Always
          lifecycle:
            postStart:
              exec:
                command: ["wg-quick",  "up", "wg0"]
            preStop:
              exec:
                command: ["wg-quick",  "down", "wg0"]
          command: ["tail",  "-f", "/dev/null"]
          volumeMounts:
            - name: wg0-key
              mountPath: /etc/wireguard/wg0.key
              subPath: wg0.key
              readOnly: true
            - name: wg0-conf
              mountPath: /etc/wireguard/wg0.conf
              subPath: wg0.conf
              readOnly: true
          ports:
            - containerPort: 51820
              hostPort: 51820
              protocol: UDP
          securityContext:
            capabilities:
              add:
                - NET_ADMIN

I set this up on a single-node K3s cluster, so I used hostPort and the Recreate strategy. On production cluster, you would want to map 51820/udp to a load balancer with a Service.

You may notice that the container is running tail -f /dev/null. This is intentional -- we're only using the pod to configure the host kernel and hold open a network namespace.

Example Dockerfile (sclevine/wg):

FROM ubuntu:focal
RUN apt-get update && \
  apt-get install -y --no-install-recommends \
    iproute2 iptables wireguard-tools && \
  rm -rf /var/lib/apt/lists/*

This Dockerfile installs the WireGuard userspace utility and setup script without the kernel module and associated dependencies.

Next, generate a key pair for the server and each peer:

$ wg genkey | tee private.key | wg pubkey > public.key

Store the server's private key in a Secret:

$ kubectl -n myvpn create secret generic wg0-key --from-file=wg0.key=./path/to/private.key

Example ConfigMap for the server config (/etc/wireguard/wg0.conf):

apiVersion: v1
kind: ConfigMap
metadata:
  name: wg0-conf
  namespace: myvpn
  labels:
    app: wireguard
data:
  wg0.conf: |
    [Interface]
    Address = 10.1.30.1/24,fdb0:5dfe:70d8:7f0b::1/64
    ListenPort = 51820
    PostUp = wg set %i private-key /etc/wireguard/wg0.key; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE; ip6tables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
    PostDown = iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE; ip6tables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
    MTU = 1500
    SaveConfig = false

    [Peer]
    # first peer
    PublicKey = <client #1 public key>
    AllowedIPs = 10.1.30.3/32,fdb0:5dfe:70d8:7f0b::3/128

    [Peer]
    # second peer
    PublicKey = <client #2 public key>
    AllowedIPs = 10.1.30.4/32,fdb0:5dfe:70d8:7f0b::4/128

This config assumes the VPN subnets will be 10.1.30.0/24 (IPv4) and fdb0:5dfe:70d8:7f0b::/64 (IPv6). I picked the private IPv4 address arbitrarily and generated the IPv6 ULA here. You may or may not need to specify the MTU. The private key (/etc/wireguard/wg0.key) is loaded dynamically from the Secret.

Client configuration:

[Interface]
PrivateKey = <client private key>
Address = 10.1.30.3/24, fdb0:5dfe:70d8:7f0b::3/64
DNS = 1.1.1.1, 1.0.0.1

[Peer]
PublicKey = <server public key>
AllowedIPs = 0.0.0.0/0, ::/0
Endpoint = <K8s node/LB IP>:51820
PersistentKeepalive = 25

To access the server, open a shell with:

$ kubectl -n myvpn exec -it deploy/wireguard bash

Check out WireGuard stats:

root@wireguard-6dbf689864-5cnxb:/# wg
interface: wg0
  public key: <server public key>
  private key: (hidden)
  listening port: 51820

peer: <peer 1 public key>
  endpoint: <peer 1 IP>:24189
  allowed ips: 10.1.30.3/32, fdb0:70d8:7f0b:5dfe::3/128
  latest handshake: Now
  transfer: 4.64 MiB received, 53.40 MiB sent

Confirm WireGuard is listening:

root@wireguard-6dbf689864-5cnxb:/# ss -lun 'sport = :51820'
State  Recv-Q   Send-Q   Local Address:Port   Peer Address:Port Process
UNCONN 0        0        0.0.0.0:51820        0.0.0.0:*
UNCONN 0        0        [::]:51820           [::]:*