OpenVPN client on dynamic IP

T

tuom

Hey,

I've been battling with this issue (or a variation thereof) for about a year now, and I can't make OpenVPN work reliably on a dynamic IP. There has to be a better way!

It took me a while to figure out what exactly was occurring, since a swift restart of OpenVPN will solve this, and initially I just kept doing that.

I have a FreeBSD box directly connected to the internet with a dynamic IP (i.e. DHCP) that acts as the OpenVPN client. It's always on, connected 24/7. I want to route everything through the tunnel so I have this set on the server:

Code: 
push "redirect-gateway def1 bypass-dhcp"

Now, obviously this will mess with the routing tables so that everything will be routed through the OpenVPN server. I've found two specific issues with this setup, I solved one of them but the other one is still eluding me:

The connection will get stuck every time dhclient pulls a new DHCP config because it flushes the routing table (removing the route to the OpenVPN server) and therefore packets will get stuck in buffers. I'm not sure if there's anything I can do from the OpenVPN config file and I can't exactly set dhclient to not update the routing tables - if the client gets moved to a completely different subnet/range the routing tables really do need to be updated.

We can clearly observe from /var/log/messages that OpenVPN goes completely nuts:
Code: 
Feb 24 10:20:03 host openvpn[48120]: TLS: tls_process: killed expiring key
Feb 24 10:20:04 host openvpn[48120]: TLS: soft reset sec=0 bytes=120594/0 pkts=628/0
Feb 24 10:20:05 host openvpn[48120]: VERIFY OK: depth=2, ...
Feb 24 10:20:05 host openvpn[48120]: VERIFY OK: depth=1, ...
Feb 24 10:20:05 host openvpn[48120]: Validating certificate key usage
Feb 24 10:20:05 host openvpn[48120]: ++ Certificate has key usage  00a0, expects 00a0
Feb 24 10:20:05 host openvpn[48120]: VERIFY KU OK
Feb 24 10:20:05 host openvpn[48120]: Validating certificate extended key usage
Feb 24 10:20:05 host openvpn[48120]: ++ Certificate has EKU (str) TLS Web Client Authentication, expects TLS Web Server Authentication
Feb 24 10:20:05 host openvpn[48120]: ++ Certificate has EKU (oid) 1.3.6.1.5.5.7.3.2, expects TLS Web Server Authentication
Feb 24 10:20:05 host openvpn[48120]: ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication
Feb 24 10:20:05 host openvpn[48120]: VERIFY EKU OK
Feb 24 10:20:05 host openvpn[48120]: VERIFY OK: depth=0, ...
Feb 24 10:20:05 host openvpn[48120]: Data Channel Encrypt: Cipher 'AES-256-CBC' initialized with 256 bit key
Feb 24 10:20:05 host openvpn[48120]: Data Channel Encrypt: Using 256 bit message hash 'SHA256' for HMAC authentication
Feb 24 10:20:05 host openvpn[48120]: Data Channel Decrypt: Cipher 'AES-256-CBC' initialized with 256 bit key
Feb 24 10:20:05 host openvpn[48120]: Data Channel Decrypt: Using 256 bit message hash 'SHA256' for HMAC authentication
Feb 24 10:20:05 host openvpn[48120]: Control Channel: TLSv1, cipher TLSv1/SSLv3 DHE-RSA-AES256-SHA, 4096 bit RSA
Feb 24 11:20:01 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:20:03 host last message repeated 5223 times
Feb 24 11:20:03 host dhclient: New IP Address (re0): <ip>
Feb 24 11:20:03 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:20:03 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:20:03 host dhclient: New Subnet Mask (re0): 255.255.248.0
Feb 24 11:20:03 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:20:03 host last message repeated 2 times
Feb 24 11:20:03 host dhclient: New Broadcast Address (re0): <ip>
Feb 24 11:20:03 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:20:03 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:20:03 host dhclient: New Routers (re0): <ip>
Feb 24 11:20:03 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:20:03 host last message repeated 570 times
Feb 24 11:20:04 host openvpn[48120]: TLS: tls_process: killed expiring key
Feb 24 11:20:04 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:20:04 host last message repeated 2230 times
Feb 24 11:20:05 host openvpn[48120]: TLS: soft reset sec=0 bytes=113432468/0 pkts=130972/0
Feb 24 11:20:05 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:20:36 host last message repeated 65594 times
Feb 24 11:21:02 host last message repeated 59661 times
Feb 24 11:21:02 host monit[818]: 'openvpn' cpu usage of 49.9% matches resource limit [cpu usage<40.0%]
Feb 24 11:21:02 host monit[818]: 'openvpn' cpu usage of 49.9% matches resource limit [cpu usage<30.0%]
Feb 24 11:21:02 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:21:04 host last message repeated 4234 times
Feb 24 11:21:05 host openvpn[48120]: TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
Feb 24 11:21:05 host openvpn[48120]: TLS Error: TLS handshake failed
Feb 24 11:21:05 host openvpn[48120]: TLS: move_session: dest=TM_LAME_DUCK src=TM_ACTIVE reinit_src=1
Feb 24 11:21:05 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:21:21 host last message repeated 35358 times
Feb 24 11:21:21 host openvpn[48120]: TLS: Initial packet from [AF_INET]<ip>:1194, sid=406a2080 a5948422
Feb 24 11:21:21 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:21:52 host last message repeated 65269 times
Feb 24 11:22:02 host last message repeated 23647 times
Feb 24 11:22:02 host monit[818]: 'openvpn' cpu usage of 49.9% matches resource limit [cpu usage<40.0%]
Feb 24 11:22:02 host monit[818]: 'openvpn' cpu usage of 49.9% matches resource limit [cpu usage<30.0%]
Feb 24 11:22:02 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:22:19 host last message repeated 37629 times
Feb 24 11:22:20 host openvpn[48120]: TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
Feb 24 11:22:20 host openvpn[48120]: TLS Error: TLS handshake failed
Feb 24 11:22:20 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:22:32 host last message repeated 28903 times
Feb 24 11:22:32 host monit[818]: Mail: Error receiving data from the mailserver 'localhost' -- Resource temporarily unavailable
Feb 24 11:22:32 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:22:35 host last message repeated 5630 times
Feb 24 11:22:35 host openvpn[48120]: TLS: Initial packet from [AF_INET]<ip>:1194, sid=e788029b bfbbb8c5
Feb 24 11:22:35 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:23:02 host last message repeated 58008 times
Feb 24 11:23:02 host monit[818]: Mail: Error receiving data from the mailserver 'localhost' -- Resource temporarily unavailable
Feb 24 11:23:02 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:23:33 host last message repeated 65354 times
Feb 24 11:23:34 host last message repeated 4458 times
Feb 24 11:23:35 host openvpn[48120]: TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
Feb 24 11:23:35 host openvpn[48120]: TLS Error: TLS handshake failed
Feb 24 11:23:35 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:24:06 host last message repeated 65823 times
Feb 24 11:24:33 host last message repeated 59082 times
Feb 24 11:24:33 host monit[818]: Mail: Error receiving data from the mailserver 'localhost' -- Resource temporarily unavailable
Feb 24 11:24:33 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:24:49 host last message repeated 37726 times
Feb 24 11:24:50 host openvpn[48120]: TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
Feb 24 11:24:50 host openvpn[48120]: TLS Error: TLS handshake failed
Feb 24 11:24:50 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:25:03 host last message repeated 29020 times
Feb 24 11:25:03 host monit[818]: Mail: Error receiving data from the mailserver 'localhost' -- Resource temporarily unavailable
Feb 24 11:25:03 host monit[818]: Alert handler failed, retry scheduled for next cycle
Feb 24 11:25:03 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:25:03 host last message repeated 2 times
Feb 24 11:25:03 host monit[818]: 'openvpn' cpu usage of 49.9% matches resource limit [cpu usage<40.0%]
Feb 24 11:25:03 host monit[818]: 'openvpn' cpu usage of 49.9% matches resource limit [cpu usage<30.0%]
Feb 24 11:25:03 host openvpn[48120]: write UDPv4: No buffer space available (code=55)
Feb 24 11:25:06 host last message repeated 6189 times
Feb 24 11:25:06 host openvpn[48120]: [UNDEF] Inactivity timeout (--ping-restart), restarting
Feb 24 11:25:06 host openvpn[48120]: SIGUSR1[soft,ping-restart] received, process restarting
Feb 24 11:25:06 host openvpn[48120]: Restart pause, 2 second(s)
Feb 24 11:25:08 host openvpn[48120]: Socket Buffers: R=[42080->42080] S=[9216->9216]
Feb 24 11:25:08 host openvpn[48120]: UDPv4 link local: [undef]
Feb 24 11:25:08 host openvpn[48120]: UDPv4 link remote: [AF_INET]<ip>:1194
Feb 24 11:26:08 host openvpn[48120]: TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
Feb 24 11:26:08 host openvpn[48120]: TLS Error: TLS handshake failed
Feb 24 11:26:08 host openvpn[48120]: SIGUSR1[soft,tls-error] received, process restarting
Feb 24 11:26:08 host openvpn[48120]: Restart pause, 2 second(s)

etc. etc.

At one point I resorted to using monit's "ping" directive to automatically restart OpenVPN when pinging fails but that's such a hack. I've also thought about automatically killing OpenVPN somehow to let monit restart it. But there has to be a better way!

Not dropping privileges and removing persist-tun might also fix this, but I'm hesitant to do that.

Here's the client config:
Code: 
client
remote <server> 1194 udp

dev tun

ca /root/pki/ovpn/root-ca.crt
cert /root/pki/ovpn/ovpnclient.crt
key /root/pki/ovpn/ovpnclient.key
tls-auth /root/pki/ovpn/hmac.key 1
remote-cert-tls server

cipher AES-256-CBC
auth SHA256

user nobody
group nobody

# persist stuff through restarts
# so privdrop works
persist-key
persist-tun

# keep resolving server infinitely
resolv-retry infinite

# do not bind to specific local port
nobind

# compress traffic
comp-lzo

# verbosity
verb 3

(For posterity, this was the second issue with this setup that I fixed by using /etc/hosts for the server address (or just use IP's in the client config): http://www.linksysinfo.org/index.php?threads/openvpn-client-persist-tun-redirect-gateway-fail.70992/)
 
How about dummying out the route updates with /etc/dhclient-enter-hooks (thanks tingo)
and fixing things permanently with
Code: 
route add default -interface tun0
route add vpnserver -interface xyz0


Juha
 
SirDice said:
There are some interesting tips here: https://openvpn.net/index.php/open-...-both-ends-of-the-connection-are-dynamic.html

Specifically the ping and ping-restart options might be of use.
Click to expand...

In the OpenVPN manual it says: "In client mode, the --ping-restart parameter is set to 120 seconds by default. This default will hold until the client pulls a replacement value from the server, based on the --keepalive setting in the server configuration. To disable the 120 second default, set --ping-restart 0 on the client."

I have "keepalive 30 120" set on the server, so that should be in effect.

Also, if I remember correctly (Ctrl+F on the manual is failing), the ping timeouts result in a "soft reset" where routes/the interface etc. don't get reinitialized.

Juha Nurmela said:
How about dummying out the route updates with /etc/dhclient-enter-hooks (thanks tingo)
and fixing things permanently with
Code: 
route add default -interface tun0
route add vpnserver -interface xyz0

Juha
Click to expand...

This might work but it seems hacky :eek:. I think another option would be to just restart OpenVPN when DHCP leases change, from the script.
 
I think I'm onto a clean solution now.

The OpenVPN manual says:
Code: 
Similarly if our IP address changes due to DHCP, we should configure our IP address change script (see man page for dhcpcd(8) ) to deliver a SIGHUP or SIGUSR1 signal to OpenVPN. OpenVPN will then reestablish a connection with its most recently authenticated peer on its new IP address.


By adding an /etc/dhclient-enter-hooks script that runs:
Code: 
#!/bin/sh
if [ $reason = "RENEW" ]; then
    kill -SIGUSR1 $(pgrep openvpn)
fi

SIGUSR1 by itself doesn't seem to work either when the routes are still defined, it won't be able to re-establish, but I guess when dhclient will refresh a lease and resets the routes, OpenVPN will be able to connect.

This is kinda hard to test though.
 
This happened again today so I was able to check this out again and check the routing table.

Indeed - the route to the OpenVPN server gets removed (dst <server_ip>/32 gw <isp_gateway>) so traffic stops flowing correctly.

Since I have persist-tun enabled, OpenVPN won't tear down the TUN device & routes when it attempts to reconnect either.

Therefore it won't be able to route to the OpenVPN server anymore since the default route (OpenVPN tunnel) is applied.

So, I guess you really might need to allow OpenVPN to run as root so it can tear down the routes and device during reconnects.

But... you could also add the route back with dhclient hooks every time or restart OpenVPN.

Related: it's not a RENEW, but BOUND that happens, I think. I added a date >> /tmp/dhcp-renew into the dhclient-enter-hooks to debug this and it wasn't a mere RENEW, so it's probably BOUND.
 
You must log in or register to reply here.
Back
Top