Vultr Private Network Configuration - Sun, May 24, 2015
for Debian/Ubuntu
Vultr.com has a cool feature where you can set up a private network between all of your instances. This provides three advantages:
- It’s more secure: services need not listen on the public interface if its only consumers are other vultr instances
- It’s really fast: gigabit ethernet
- It’s unmetered: it doesn’t count toward your bandwidth quota
This article is about how to configure it on Debian or Ubuntu.
As root, we’ll create a new file in /etc/network/interfaces.d. You can name it whatever you want; I named it private. It’s a good idea to put a new file in interfaces.d rather than edit /etc/network/interfaces directly. This way, during upgrades, you don’t have to merge files if the interfaces file changes. Make this file on each server, being careful to assign a unique IP address (one is suggested on the dashboard).
hut8@lorenz ~ % cat /etc/network/interfaces.d/private
auto eth1
iface eth1 inet static
address 10.99.0.10
netmask 255.255.0.0
mtu 1450
As they note, you can actually use any private IP address that you want. They will only be visible to other instances in your account. Just make sure they’re all unique and use something in the private address space.
Then apply the change:
hut8@lorenz ~ % sudo /etc/init.d/networking restart
That will not close your current connections, so you can do that over SSH.
Speed test
hut8@lorenz ~ % iperf -n 1024M -c crib ------------------------------------------------------------
Client connecting to crib, TCP port 5001
TCP window size: 45.0 KByte (default)
------------------------------------------------------------
[ 3] local 10.99.0.10 port 36032 connected with 10.99.0.11 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-12.5 sec 1.00 GBytes 685 Mbits/sec
Crazy fast!
MTU setting
I was pretty disappointed at first when the MTU was set to (the default) 1500:
hut8@lorenz ~ % iperf -n 1024M -c
------------------------------------------------------------
Client connecting to crib, TCP port 5001
TCP window size: 45.0 KByte (default)
------------------------------------------------------------
[ 3] local 10.99.0.10 port 30123 connected with 10.99.0.11 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-779.6 sec 1.00 GBytes 11.0 Mbits/sec
These vultr.com docs don’t mention the MTU issue, although on the dashboard there’s a mention of it if you click a button for more information.
As you can see, a difference of 6200%!
What accounts for this massive difference? IP Fragmentation. The VPN behind the private networking adds a little bit of data as a header on every packet. The MTU of the network is probably 1500. Then, when the OS sends a packet that’s 1500 bytes, the overhead from the VPN pushes the packet size just over the MTU. That causes the packet to be fragmented (outside of the server), then all kinds of overhead ensues. So keep your MTU for that interface at 1450.