post: add rpi-wireguard-iptables
Prithu Goswami pg@prithu.dev
Wed, 01 Feb 2023 12:42:15 +0530
3 files changed,
350 insertions(+),
136 deletions(-)
D
content/posts/rpi-wireguard-iptable.md
@@ -1,136 +0,0 @@
---- -title: "RPi server, wireguard mesh and iptable" -date: 2023-01-06T00:23:53+05:30 -draft: true ---- - -A few months ago, I wanted to host files directly from a 1TB hard disk on the -public net. These files were just some photos and videos that I wanted to share -with some peeps. Nothing fancy, just a webdav server using -[rclone](https://rclone.org/commands/rclone_serve_webdav/) would do—a -very ad-hoc setup as you can see. I wanted it to be reachable at -`share.prithu.dev` to give it a short friendly remeberable name. I used the RPi -4 to be the host. I attached the 1TB hard drive to one of it's usb ports, -`mount`'d the drive at `/mount/hdd` and ran: - -``` -rclone serve webdav --user foo --pass secret --addr 0.0.0.0:8080 /mount/hdd/shared-dir -``` - -This is simple enough and now all I need to do is, using port-forwarding rules -on my router, expose this port to the world and and then point -`share.prithu.dev` to my home's IP address, and bob's your uncle … or is he? -Well, the glaring problem here is me exposing my home ip—which is, to say -the least, a bad practice and something that I want to avoid. Also, there's the -problem of dynamic IPs of home broadband, but that's a fairly easy problem to -solve by writing a script[^1] to update the dns records every X minutes. - -[^1]: something like this: https://git.prithu.dev/dotfiles/blob/master/bin/scripts/update_home_dns.sh - -I can setup a VPS with a public facing port that I can use as a proxy between -the RPi and the internet. I can listen on the VPS and forward the connection to -the RPi. The VPS acts as a relay server and routes packets to the rpi. One way -to do this is using good ol' ssh tunnels (port forwarding) using the `-R` -option to forward the remote port to my RPi. In my experience however, I have -found ssh tunnels to be a little unreliable, especially when it comes to spotty -home broadband connection; due to which the connection breaks and you will have -to either restart the client or write a script or a service to restart the -client automatically. I wanted the server to be reliablly hosted for as long as -it was required with minimum to no maintainance from my side so I thought to -try my hands on wireguard and in the process also learn more about it. - - -## The final setup - -This is how the final setup is going to be: - -{{< figure src="/img/mesh-setup.png" title="setup of rpi with vps" width="100%" >}} - -Here's the wireguard network consisting of the RPi and VPS. I also have my laptop -and phone connected as well, but I left them out of the diagram for simplicity. -The devices are assigned private IP addresses in the subnet of `10.8.0.0/24`. -(an arbitrary choice; something different from the usual `10.0.0.0/24`) - -## What is Wireguard? - -[Wireguard](https://www.wireguard.com/), simply put, allows you create VPN -tunnels. You can also connect multiple hosts (Peers) and form a mesh of -interconnected devices within your private network. -Wireguard is very easy to setup, infact, it's one of their aims to be as easy -to setup as SSH. Here's a quote from the wireguard homepage: - -> WireGuard aims to be as easy to configure and deploy as SSH. A VPN connection -> is made simply by exchanging very simple public keys – exactly like -> exchanging SSH keys – and all the rest is transparently handled by WireGuard. -> It is even capable of roaming between IP addresses, just like Mosh. There is -> no need to manage connections, be concerned about state, manage daemons, or -> worry about what's under the hood. WireGuard presents an extremely basic yet -> powerful interface. - - - - - -``` -Cover: - -- add config for each device (laptop, phone, rpi and vps) to explain the mesh - network and have a small diagram of the mesh network -- mini ups with link -- one photo of the whole setup - -The Class A range (`10.0.0.0 to 10.255.255.255`) is the most common one used. -It's easier on the eyes and also easy to remember (I mean what would you rather -prefer—`10.5.0.1` or `172.16.5.1`?); You also have a big range of values -to choose from. - - -[Interface] -Address = 10.24.0.1/24 -ListenPort = 52499 -PrivateKey = 6HTkKeOeHLb92PPQ4DYBNUC/TSZqqJLFXe8zWEEAf0Q= -PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o %i -j ACCEPT; iptables -t nat -A POSTROUTING -o ens3 -j MASQUERADE -PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -D FORWARD -o %i -j ACCEPT; iptables -t nat -D POSTROUTING -o ens3 -j MASQUERADE - -[Peer] -PublicKey = UB8PUnmfnF2e54651NF1rAbOcB/Ud5dVq49rJd4BNzE= -AllowedIPs = 10.24.0.3/32 -Endpoint = 122.171.44.71:54035 - -[Peer] -PublicKey = kZPtUBJWwTNAkLWtfRxoe20UYToAV3brFBxbJRorEG4= -AllowedIPs = 10.24.0.4/32 - -[Peer] -PublicKey = COCePzCqoQHaaTlbniqGUJIZwpzRwTWjoaAt/puTc3g= -AllowedIPs = 10.24.0.2/32 - - - -links: - -``` -## Setting up the wireguard mesh - -### On the VPS - -1. **Install wireguard**. Wireguard is [available pretty much - everywhere](https://www.wireguard.com/install/). You can also [compile it - from source](https://www.wireguard.com/compilation/). The instructions on - the website are pretty straighforward and easy to follow. On my ubuntu VPS - it's just: - - ``` - sudo apt install wireguard - ``` - -2. **Generate keypair**. Wireguard works with simple public and private key pair. - Each peer in the wireguard network has a public-private key pair associated - with it. The keys are what the pees use to authenticate themsevles and - encrypt the packets. the `wg` utility can be used to generate these key - pairs `setukp the` - - ``` - umask 077 - wg genkey | tee privatekey | wg pubkey > publickey - ```
A
content/posts/rpi-wireguard-iptables.md
@@ -0,0 +1,350 @@
+--- +title: "RPi server behind VPS using wireguard and iptables NAT" +date: 2023-02-01T09:23:53+05:30 +draft: true +description: Documenting the process of setting up wireguard on a VPS and forwarding traffic to a host on the private network using iptables +--- + +A few months ago, I wanted to host files stored on a portable hard disk on the +public internet. These files were just some photos and videos that I wanted to +share with some of my cousins. Nothing fancy, just a webdav server using +[rclone](https://rclone.org/commands/rclone_serve_webdav/) would +suffice—a very ad-hoc setup as you can see. I wanted it to be reachable +at `share.prithu.dev` to give it a short and friendly rememberable name. I used +the RPi 4 to be the host machine, attached the hard drive to one of it's usb +ports, `mount`'d the drive at `/mount/hdd` and ran: + +``` +rclone serve webdav --user foo --pass secret --addr 0.0.0.0:8080 /mount/hdd/shared-dir +``` + +This is simple enough. Now, all I need to do is, using port-forwarding rules on +my router, expose this port to the world and point `share.prithu.dev` to my +home IP address, and bob's your uncle … or is he? Well, the glaring problem +here is me exposing my home IP address—which is, to say the least, a bad +practice and something that I want to avoid. Also, there's the problem of +dynamic IP of home broadband, but that's a fairly easy problem to solve by +writing a script to update the DNS records every X minutes.[^1] + +[^1]: something like [this script](https://git.prithu.dev/dotfiles/blob/master/bin/scripts/update_home_dns.sh) + +Another solution is I can use one of my VPSs with a static public IP as a proxy +between the RPi and the internet—effectively hiding my home IP from the +internet. I will also not have to worry about the changing home IP. One way to +do this is using good ol' ssh tunnels (port forwarding) and forward the remote +port to my RPi. In my experience however, I have found ssh tunnels to be a +little unreliable, especially when it comes to spotty home broadband +connection, due to which the connection breaks and you will have to either +restart the client or write a script or a service to restart the client +automatically. SSH tunnels are good for quick one-off tasks though. They can be +a life saver—that is, If you can remember the right syntax of `-R` and `-L` +options without referring to the man page (now that's a challenge!). I wanted +the server to be reliably hosted for as long as it was required with minimum to +no maintenance from my side. So, I thought to try my hands on wireguard with +NAT on the VPS. The idea is to make the VPS act as a relay server which routes +packets to the RPi (sort of like "your router in the cloud" if you +will)—while the RPi server only being reachable from within the wireguard +network. + + +## The final setup + +This is how the final setup is going to be: + +{{< figure src="/img/mesh-setup.png" width="100%" >}} + +Here's the wireguard network consisting of the RPi and VPS. The devices are +assigned private IP addresses in the subnet of `10.8.0.0/24`. + + +## Setting up the wireguard network + +[Wireguard](https://www.wireguard.com/) makes it super simple to setup a VPN. +In fact, it's one of its aims to be as easy to setup as SSH: + +> WireGuard aims to be as easy to configure and deploy as SSH. A VPN connection +> is made simply by exchanging very simple public keys – exactly like +> exchanging SSH keys ... + +I'll try to be brief with this part, at the same time, try explaining most of +it (for documentation sake for my future self) but more info can be found in +this excellent resource: [unoffical wireguard +doc](https://github.com/pirate/wireguard-docs). + + +**1. Install wireguard**. Wireguard is [available pretty much +everywhere](https://www.wireguard.com/install/). You can also [compile it from +source](https://www.wireguard.com/compilation/). The instructions on the +website are pretty straight forward and easy to follow. On my ubuntu VPS (and +the RPi) it's just: + +``` +$ sudo apt install wireguard +``` + +**2. Generate the key pair**. Wireguard works with simple public-private key +pairs—just like ssh. Each peer in the wireguard network has a public key +associated with it. The `wg` utility is used to generate these key pairs: + +``` +umask 077 +wg genkey | tee privatekey | wg pubkey > publickey +``` + +**3. Configure wireguard**. The configuration file for a network is located at +`/etc/wireguard/*.conf` by default. You can setup and configure multiple +wireguard networks—each having it's own `*.conf` file. For example for a +network called `wg0`, the config file is `/etc/wireguard/wg0.conf`. + +- **On the VPS** + + ``` + [Interface] + Address = 10.8.0.1/24 + ListenPort = 42069 + PrivateKey = <privatekey of VPS> + + [Peer] + PublicKey = <publickey of RPi> + AllowedIPs = 10.8.0.2/32 + + # Peer 2 for example + [Peer] + PublicKey = <publickey of peer 2> + AllowedIPs = 10.8.0.3/32 + ``` + + `Address` is the IP address associated with the VPS in the network. and the + `/24` suffix means that the VPS can also route traffic not just for itself + but also for the whole range of `10.8.0.0/24`. The VPS is essentially + performing the role of a bounce server that routes traffic to/from all the + other peers in the network. `ListenPort` is the port it listens on for + connections from peers. This port needs to be publicly reachable, hence punch + a hole through the firewall and update the networking security rules of the + VPS (something that is easily forgotten). `PrivateKey` is the contents of the + `privatekey` file. + + Each `[Peer]` section defines a peer in the network. `PublicKey` is the + public key of the peer—in this case, the RPi's `publickey` file. + `AllowedIPs` is simply the IP address range the peer will route traffic for. + In this case, it's a single IP—indicated by the `/32` suffix—of + `10.8.0.2`. This will be the address of the RPi within the network. More + `[Peer]` sections can be added to add hosts to the network and they'll all be + able to reach each other by routing traffic through the bounce server—forming + a virtual mesh network. + + +- **On the RRi** + + ``` + [Interface] + Address = 10.8.0.2/32 + PrivateKey = <privatekey of RPi> + + [Peer] + PublicKey = <publickey of VPS> + AllowedIPs = 10.8.0.1/24 + Endpoint = 203.0.113.42:42069 + PersistentKeepalive = 25 + ``` + + `Address` is the IP address that the RPi associates itself within the + network. Note the `/32` suffix means that only a single IP is associated with + this host. It tells the network, not a range, but a single IP is reachable by + this node. `PrivateKey` is the contents of the `privatekey` file generated on + the RPi. The `[Peer]` section here represents the bounce server (the VPS). + `PublicKey` is the `publickey` of the VPS. `Endpoint` is the publicly + reachable IP and port of the peer—this is the public IP of the VPS and + the `ListenPort` from the config on the VPS. `PersistentKeepalive` is + required for peers behind a typical home router performing NAT. It keeps the + connection open in order to be able to receive incoming packets even after a + long interval of inactivity.[^b] + + [^b]:more about it [here](https://www.wireguard.com/quickstart/#nat-and-firewall-traversal-persistence) + +**4. Start wireguard**. + +``` +sudo wg-quick up wg0 +``` + +This will bring up the `wg0` network interface and setup the link. You can also +use the systemd service file that comes with the package: + +``` +sudo systemctl start wg-quick@wg0 +``` +Run `wg` to see if everything is working as expected. + +``` +prithu@rpi:~ $ sudo wg +interface: wg0 + public key: SomePublickeyForRPi6VzxPWt8sdIj8rkjmzMsxjm8= + private key: (hidden) + listening port: 52912 + +peer: SomePublicKeyForVPS+px4kBLmGzIrBra2K2Rc3yCw= + endpoint: 203.0.113.42:42069 + allowed ips: 10.8.0.0/24 + latest handshake: 27 seconds ago + transfer: 265.41 MiB received, 256.75 MiB sent + persistent keepalive: every 25 seconds +``` + +The RPi should be reachable at `10.8.0.2` from the VPS and the VPS at +`10.8.0.1` from the RPi. You should be able `ping` the other machine to test +the connection. + +## Forwarding packets from the VPS + +Now that I have the RPi and the VPS in the same wireguard network, I just need +to forward traffic hitting port `80` on the VPS to port `8080` on my RPi, where +the rclone process is listening on. I could simply use a reverse proxy like +nginx or haproxy, but I wanted to make this a learning opportunity and go a +level lower, closer to the networking stack and try using `iptables` and it's +NAT capabilities. + +### A little about `iptables` + +`iptables` uses [netfilter](https://www.netfilter.org/) underneath which is a +framework for packet filtering, mangling, rate limiting, logging and lots +more, baked right into the Linux kernel. Netfilter is a kernel-space program +and exposes hooks that let you interact with network packets and perform +actions on them. `iptables` is an easy to use user-space utility that provides +an interface to netfilter and let's you create rules and actions that let you +control the fate of each packet that enters or leaves the host. [Here's an +excellent +article](https://www.digitalocean.com/community/tutorials/a-deep-dive-into-iptables-and-netfilter-architecture) +I referred that goes through the architecture of netfilter and iptables. There +are myriad of articles and guides explaining how to use iptables and its syntax +of rule-specifications; so I am not going into detail on that. + +### NAT'ing the packets + +First, lets setup rules in the `PREROUTING` and `POSTROUTING` chain in the +`nat` table. `PREROUTING` chain in the `nat` table gives us access to the +`DNAT` jump target that let's you change the destination of a packet entering +the host. Similarly, `POSTROUTING` chain in the `nat` table gives us access to +the `SNAT` jump target that lets you change the source of the packet leaving +the host. + +``` +sudo iptables -t nat -A PREROUTING -i ens3 -p tcp --dport 80 -j DNAT --to-destination 10.8.0.2:8080 +sudo iptables -t nat -A POSTROUTING -o wg0 -p tcp -d 10.8.0.2 --dport 8080 -j SNAT --to-source 10.8.0.1 +``` + +The first rule changes the destination address of every TCP packet (`-p tcp`) +hitting port `80` (`--dport 80`) on the host to `10.8.0.2` and the port to +`8080`. The `-i ens3` specifies the interface for the rule, which is the public +interface of the VPS. Before the packet leaves the VPS destined to the RPi (`-d +10.8.0.2 --dport 8080`), the second rule changes its source address to the +address of the VPS in the wireguard network (`10.8.0.1`). This is so that the +RPi replies back to the VPS instead of replying to the original client's +address. The `-o wg0` argument specifies the rule to be applied only for +packets going out through `wg0` wireguard interface. + +{{< aside >}} + +#### Note on MASQUERADE + +`MASQUERADE` is a special target that is only available in the `POSTROUTING` +chain of the `nat` table. It's a specialized case of `SNAT` target and +according to the docs it should only be used for dynamically-assigned IP +addresses. i.e if the interface goes down and comes back up with a new IP. In +our case though we have a static IP (`10.8.0.1`) assigned to the host. I am +sure there are good reasons why not use it if you already have a static ip +assigned and don't expect it to change; iptables might be performing extra +steps for `MASQUERADE` targets thus might add some overhead. However, you could +use it with static IP if you are okay with the added overhead. If we were to +use `MASQUERADE` though, we would not have to mention `--to-source` argument from +the `POSTROUTING` rule above. + +``` +sudo iptables -t nat -A POSTROUTING -o wg0 -j SNAT MASQURADE +``` + +{{< /aside >}} + +### Allowing packets to be forwarded + +After being processed by the `PREROUTING` chain the packet goes through the +`FORWARD` chain. Here's what the sequence looks like: + +``` + .------------. .---------. .-------------. +ens3--->| PREROUTING +--->| FORWARD +--->| POSTROUTING +---> wg0 (to rpi) + '------------' '---------' '-------------' +``` + +Usually the default policy of the `FORWARD` chain is to `DROP` the packet. We +need to allow packets to be forwarded. The following two rules allow packets to +be forwarded across the two interfaces: + +``` +sudo iptables -A FORWARD -i ens3 -o wg0 -j ACCEPT +sudo iptables -A FORWARD -o wg0 -i ens3 -j ACCEPT +``` + +{{< aside >}} + +Although the above rules work fine, I could be more specific by including the +destination address, port and protocol—making sure we only allow the traffic we +want to forward—as a good security measure. But since this is a personal VPS, I +don't mind being a little lax with the rules. Here are more specific rules we +can use instead: + +``` +sudo iptables -A FORWARD -i ens3 -o wg0 -d 10.8.0.2 -p tcp --dport 8080 -m conntrack --ctstate NEW -j ACCEPT +sudo iptables -A FORWARD -i ens3 -o wg0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT +sudo iptables -A FORWARD -i wg0 -o ens3 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT +``` + +`conntrack` allows us to use the `--ctstate NEW` option to allow new +connections through at first. Next we use the `ESTABLISHED` and `RELATED` to +allow any packets that are part of an already established or related +connection. + +{{< /aside >}} + + + +Almost forgot—we need to also enable traffic forwarding at the kernel level by +enabling the `net.ipv4.ip_forward` kernel parameter, which is disabled by +default. + +``` +sysctl -w net.ipv4.ip_forward=1 +# to persist across reboot +echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf +``` + +## + +That's it! Now, all I need to make sure is that port `80` is reachable on the +VPS and point `share.prithu.dev` to the VPS's public IP. Visiting +`http://share.prithu.dev` will render the rclone webdav server's directory +listing page. + +I can use this setup for various other self-hosting projects in the +future—especially when I want to make sure I have the physical access to the +machine providing the service. Forwarding more ports/protocol can be done +simply by copying the same rules (`PREROUTING`, `POSTROUTING`) from above and +changing the `--dport` and `-p` arguments accordingly and making sure they are +allowed in the `FORWARD` chain. All in all, this was a good learning +opportunity—especially learning about netfilter and `iptables`. I thought +`iptables` was just used as a firewall, while it's a lot more than just that +and is a pretty versatile tool. While learning about iptables I also came +across [nftables](https://www.netfilter.org/projects/nftables/index.html) which +is a modern successor of iptables which is more flexible and performant. + +## Uninterupted Power Supply for the RPi + +One last thing to add is an UPS. The RPi was powered by an AC adapter, which is +not ideal if you are running it as a server since power outages may cause +uptime disruptions, even worse, disk corruptions—not something we want while +running a server. In order to prevent that, I ordered [this +micro-UPS](https://www.amazon.in/gp/product/B09XBBZKDC/ref=ppx_yo_dt_b_asin_title_o09_s00?ie=UTF8&th=1) +to keep my RPi up for enough time until power came back on. This UPS has the +correct rating (5V 3A) required for the RPi and has a capacity of 7500 mAh. The +UPS seamlessly switches over to battery power without disrupting the Pi. It's a +life saver and keeps the RPi up reliably during outages.
