OpenVPN Client in a jail

cschiewek

New Member

Reaction score: 2
Messages: 13

Hello all,

Host is FreeBSD 9.1/AMD64

I'd like to create a jail and have all traffic within said jail routed over an OpenVPN connection, but have traffic outside of the jail going straight out.

I've tried it using straight up IP alias and that didn't work. I've found examples online of using virtual networks inside a jail using VIMAGE (http://wiki.polymorf.fr/index.php/Howto:FreeBSD_jail_vnet), but from what I gather VIMAGE drivers were removed in 9.1.

I'm a bit of a networking n00b, so any advice on what direction I should pursue would be awesome.

TIA,

Curtis
 

myst3r10n

New Member


Messages: 11

I have a similar problem. My network connection between the main system and the jail works well, the internet connection too on both. But if the OpenVPN client is activated on the main system then I cannot connect to the internet from inside the jail. I can only connect from main system to internet over the OpenVPN client configuration but not within the jail.
 

DaCa

New Member

Reaction score: 5
Messages: 4

I also wanted to do this, just assign the VPN to one specific jail and have regular networking on the host and other jails. After a lot of experimentation I found a way to do it. The trick is to do source routing, which can be done with the help of PF.

First, you need to unhide your tun interface in your jail's devfs rules:

Code:
add path tun0 unhide
You also have to avoid that OpenVPN adds routing rules to redirect all traffic to the VPN and replace them with your own: one routing rule to make it possible to reach the gateway on the remote side, and two PF rules to route the jail's traffic in both directions through the tunnel. I did it this by adding the following to my openvpn.conf

Code:
script-security 2
route-noexec
route-up /usr/local/etc/openvpn/route-up.sh
First is to allow scripts, second to tell it to not add routing rules itself, and third is our replacement script: (the variables are provided by OpenVPN through the environment)

Code:
#!/bin/sh

/sbin/route add -net $route_network_1 $route_net_gateway 255.255.255.255
/sbin/pfctl -a vpn -f - <<!
pass in reply-to (tun0 $route_vpn_gateway) to $ifconfig_local
pass out route-to (tun0 $route_vpn_gateway) from $ifconfig_local
!
and in your pf.conf add an anchor to include these rules:

Code:
anchor vpn quick
 
OP
OP
cschiewek

cschiewek

New Member

Reaction score: 2
Messages: 13

DaCa:

Are you running OpenVPN from within the jail or from outside? I've tried following your instructions and when I fire up OpenVPN from within the jail I get the following error:

Code:
openvpn: writing to routing socket: No such process
I can't seem to add the routes from within the jail.

I then assumed that you were running OpenVPN outside the jail and just routing all jail traffic over the tun0 device, but I can't get that to work either.

Any idea?
 

myst3r10n

New Member


Messages: 11

This not working for me. I always get:
Code:
n Jul  7 12:10:02 2013 us=72489 Could not retrieve default gateway from route socket:: No such process (errno=3)
Sun Jul  7 12:10:02 2013 us=72498 ROUTE: default_gateway=UNDEF
Sun Jul  7 12:10:02 2013 us=72562 TUN/TAP device /dev/tun0 opened
Sun Jul  7 12:10:02 2013 us=72572 ioctl(TUNSIFMODE): Device busy: Device busy (errno=16)
Sun Jul  7 12:10:02 2013 us=72587 do_ifconfig, tt->ipv6=0, tt->did_ifconfig_ipv6_setup=0
Sun Jul  7 12:10:02 2013 us=72603 /sbin/ifconfig tun0 10.0.33.46 10.0.33.45 mtu 1500 netmask 255.255.255.255 up
ifconfig: ioctl (set mtu): Operation not permitted
ifconfig: up: permission denied
Sun Jul  7 12:10:02 2013 us=73543 FreeBSD ifconfig failed: external program exited with error status: 1
Sun Jul  7 12:10:02 2013 us=73554 Exiting due to fatal error
 
OP
OP
cschiewek

cschiewek

New Member

Reaction score: 2
Messages: 13

@myst3r10n:

That's easy to fix. You need to create the tun device outside of the jail.

To create it on the fly run ifconfig tun create. To create it at startup, add the following to rc.conf:
Code:
cloned_interfaces="tun"
 
Last edited by a moderator:

DaCa

New Member

Reaction score: 5
Messages: 4

Yes, it is important to note that you have to run the OpenVPN software on the host and not in the jail. This is because within a jail you cannot configure network interfaces or the routing table. Everything I mentioned in my message should be done on the host and not in a jail. With that setup, and after starting OpenVPN you should be able to start a jail with the VPN's IP address.
 

myst3r10n

New Member


Messages: 11

If the OpenVPN is running on the host then every traffic goes over OpenVPN. My Idea is more, that you have a jail. And on that jail run OpenVPN and every traffic from the jail goes over OpenVPN. But the traffic from the host goes not over OpenVPN. Is this with your installation possible or have I misunderstood?
 

DaCa

New Member

Reaction score: 5
Messages: 4

Yes, this is exactly what I do here. Normally OpenVPN adds routes so all traffic goes over the VPN. I tell it to not do that and inject my own routes so only the jail's traffic gets routed over it.
 
OP
OP
cschiewek

cschiewek

New Member

Reaction score: 2
Messages: 13

So I've set it up as instructed, but my jail traffic is not going over the VPN (verified by going to whatismyip.com).

Here's what my setup looks like:

rc.conf
Code:
# Network setup
hostname="server"
ifconfig_re0="inet 192.168.1.3/24"
ifconfig_re0_ipv6="inet6 accept_rtadv"
ifconfig_re0_alias0="inet 192.168.1.4/24"
cloned_interfaces="tun"
defaultrouter="192.168.1.1"
pf_enable="YES"
openvpn_enable="YES"
openvpn_configfile="/usr/local/etc/openvpn/default.ovpn"

# Jails
jail_enable="YES"
jail_list="vpn"
jail_vpn_rootdir="/usr/jails/vpn"
jail_vpn_hostname="vpn.server"
jail_vpn_ip="192.168.1.4"
jail_vpn_devfs_enable="YES"
jail_vpn_devfs_ruleset="devfsrules_jail_vpn"
default.ovpn
Code:
client
dev tun
proto udp
remote *.*.*.* 443
remote *.*.*.* 443
remote-random
resolv-retry infinite
nobind
ns-cert-type server
cipher AES-256-CBC
comp-lzo
verb 3
explicit-exit-notify 5
ca "ca.crt"
cert "user.crt"
key "user.key"

script-security 2
route-noexec
route-up /usr/local/etc/openvpn/route-up.sh
route-up.sh
Code:
#!/bin/sh

/sbin/route add -net $route_network_1 $route_net_gateway 255.255.255.255
/sbin/pfctl -a vpn -f - <<!
pass in reply-to (tun0 $route_vpn_gateway) to $ifconfig_local
pass out route-to (tun0 $route_vpn_gateway) from $ifconfig_local
!
And my devfs rules and pf.conf include only the configs mentioned. Thoughts?
 

myst3r10n

New Member


Messages: 11

Code:
client
dev tun
proto udp
remote *.*.*.* 443
remote *.*.*.* 443
remote-random
resolv-retry infinite
nobind
ns-cert-type server
cipher AES-256-CBC
comp-lzo
verb 3
explicit-exit-notify 5
ca "ca.crt"
cert "user.crt"
key "user.key"

script-security 2
route-noexec
route-up /usr/local/etc/openvpn/route-up.sh
The option route-noexec doesn't work for me.
 

myst3r10n

New Member


Messages: 11

myst3r10n said:
The option route-noexec doesn't work for me.
I could fix that issue and my configuration works now too. I did the same like DaCa and have connection between jail and VPN. But, I have still a problem with the IP addresses. For example, my jail configuration has:
Code:
jail_vpn_ip_multi0="tun0|10.0.51.66 10.0.51.65 mtu 1500 netmask 255.255.255.255
But no jail_vpn_ip= exists in my jail configuration because that not works. My both IP addresses from jail_vpn_ip_multi0= are from OpenVPN:
Code:
Tue Jul 16 04:50:51 2013 us=845743 TUN/TAP device /dev/tun0 opened
Tue Jul 16 04:50:51 2013 us=845756 do_ifconfig, tt->ipv6=0, tt->did_ifconfig_ipv6_setup=0
Tue Jul 16 04:50:51 2013 us=845772 /sbin/ifconfig tun0 10.0.51.66 10.0.51.65 mtu 1500 netmask 255.255.255.255 up
It only works with these IP addresses. But these both IP addresses changes every time if OpenVPN is restarted. You could imagine that our manual configuration of jail_vpn_ip_multi0= is not a good solution.

My question is now, exists a way to do this with automation?
 

DaCa

New Member

Reaction score: 5
Messages: 4

Yes, you can. Set a name to the jail so you can refer to it:

Code:
jail_vpn_flags="-n vpn"
Now you can add to the end of your route_up.sh

Code:
/usr/sbin/jail -m name=vpn ip4.addr=$ifconfig_local
This works for one IP address, maybe it needs some adaption because you are getting two adresses.
 

myst3r10n

New Member


Messages: 11

Thank you @DaCa, it works now like a charm :)

I have written for my tun0 device two IPs:

Code:
/usr/sbin/jail -m name=play ip4.addr="$ifconfig_local,$ifconfig_remote"
I have experimented with multiple instances of OpenVPN. It's well explained at the top of the /usr/local/etc/rc.d/openvpn file. I had a successful setup of multiple OpenVPN instances on my VirtualBox-based virtual machine with bridged network interfaces (bridge0/tap0).

Next I will try it on my host system for multiple jails ;)
 
Last edited by a moderator:

myst3r10n

New Member


Messages: 11

I still have a question regarding the random IP address for my tun0 interface. It's a little annoying. Is it possible to create a second tun1 interface with a fixed IP address and redirect all traffic from tun1 to tun0? E.g. using pf.conf rules?
 

myst3r10n

New Member


Messages: 11

I have solved my issue.

I created two tun devices. A tun0 for my jail with fixed IP address. And tun1 for OpenVPN, where I have unfortunately always another IP address.

The jail configuration looks like:

Code:
#jail_www_ip="192.168.1.2 <- hide them
jail_www_ip_multi0="tun0|10.0.1.2 10.0.2.2 mtu 1500 netmask 255.255.255.255"
/etc/rc.conf.d/openvpn_www contains:

Code:
openvpn_www_enable="YES"
openvpn_www_flags="--dev tun1"
The route-up.sh script has been extended:

Code:
#!/bin/sh

jail="www"
ifconfig_jail=`dig +short www.example.org` # <- Get 10.0.1.2 of tun0

/sbin/route add -net $route_network_1 $route_net_gateway 255.255.255.255
/sbin/pfctl -a ${jail}_vpn -f - <<!
pass in on re0 reply-to ($dev $route_vpn_gateway) to $ifconfig_local
pass out on re0 route-to ($dev $route_vpn_gateway) from $ifconfig_local
!
/usr/sbin/jail -m name=$jail interface=tun0 ip4.addr="$ifconfig_local,$ifconfig_remote,$ifconfig_jail"
You could write your jail IP directly into $ifconfig_jail too. But, I like more the named solution together with dig. And notice that $ifconfig_jail must be within ip4.addr. Else the network connection to jail wouldn't work.

Well, that's it :)
 

myst3r10n

New Member


Messages: 11

A hint to solve the startup conflicts at boot time between openvpn and your jail.

Just use in your /etc/rc.conf.d/jail file:

Code:
jail_www_exec_poststart0="service openvpn_www onestart"
jail_www_exec_prestop0="service openvpn_www stop"
If any other service requires your openvpn connection then just extend:

Code:
jail_www_exec_poststart0="service openvpn_www onestart"
jail_www_exec_poststart1="jexec www service MYSERVICE onestart"
jail_www_exec_prestop0="jexec www service MYSERVICE stop"
jail_www_exec_prestop1="service openvpn_www stop"
And ensure that your MYSERVICE have a sleep 5 or something similar. Because openvpn takes a little time until a connection is established.

Another approach would be to use
Code:
# REQUIRE: www jail
in your /usr/local/etc/rc.d/openvpn_www file.
 

FreeBSDNewUser

New Member


Messages: 6

I could fix that issue and my configuration works now too. I did the same like DaCa and have connection between jail and VPN. But, I have still a problem with the IP addresses. For example, my jail configuration has:
Code:
jail_vpn_ip_multi0="tun0|10.0.51.66 10.0.51.65 mtu 1500 netmask 255.255.255.255
But no jail_vpn_ip= exists in my jail configuration because that not works. My both IP addresses from jail_vpn_ip_multi0= are from OpenVPN:
Code:
Tue Jul 16 04:50:51 2013 us=845743 TUN/TAP device /dev/tun0 opened
Tue Jul 16 04:50:51 2013 us=845756 do_ifconfig, tt->ipv6=0, tt->did_ifconfig_ipv6_setup=0
Tue Jul 16 04:50:51 2013 us=845772 /sbin/ifconfig tun0 10.0.51.66 10.0.51.65 mtu 1500 netmask 255.255.255.255 up
It only works with these IP addresses. But these both IP addresses changes every time if OpenVPN is restarted. You could imagine that our manual configuration of jail_vpn_ip_multi0= is not a good solution.

My question is now, exists a way to do this with automation?
Where do you place the jail_vpn_ip_multi0="tun0|10.0.51.66 10.0.51.65 mtu 1500 netmask 255.255.255.255" is that going in the rc.conf for the host system? this thread really jumps around and is hard to follow could you possibly paste the steps taken that achieve the end result? I could really use some help on this as I have been going over this page for the last day now and cannot get this working I keep getting this in my OpenVPN log file:
Code:
route: writing to routing socket: Network is unreachable
add net 192.168.0.1: gateway 255.255.0.0 fib 0: Network is unreachable
No ALTQ support in kernel
ALTQ related functions disabled
stdin:3: syntax error
pfctl: Syntax error in config file: pf rules not loaded
Sun Feb  1 20:15:11 2015 WARNING: Failed running command (--route-up): external program exited with error status: 1
 

darrendavid

Member

Reaction score: 1
Messages: 23

Resurrecting an ancient thread here because I'm in the midst of trying to do the same thing on FreeBSD 11-RELEASE right now and hitting a bit of a snag with the VPN startup. It seems that the route-up.sh script is failing to add the route because it seems that $route_gateway_1 is not set. I'm getting:

route: writing to routing socket: Network is unreachable
add net 10.0.1.1: gateway 255.255.255.255 fib 0: Network is unreachable

I edited the script to output the values of the main variables and I get:

route_network_1=
route_net_gateway= 10.0.1.1

So the script is failing to set the route properly, and I'm not sure why. I want to try to do it manually but I'm not sure what the value of route_network_1 /should/ be. Anyone else hit this snag?

Thanks in advance.
 
Top