OpenVPN, PF and FreeBSD 9.2

OK I am trying again to get OpenVPN to allow Internet traffic from the client to pass through the server. To support this effort I have installed FreeBSD 9.2 on a virtual machine in VirtualBox. I have this testbed up and operating with minimal applications with full Internet connectivity. I have installed OpenVPN 2.3.1 and have it running and can access the virtual server from a client. The server.conf is:
Code:
port 1194
proto tcp
dev tun
ca /usr/local/etc/ssl-admin/active/ca.crt
cert /usr/local/etc/ssl-admin/active/server.crt
key /usr/local/etc/ssl-admin/active//server.key
dh /usr/local/etc/ssl-admin/dh2048.pem
server  10.8.0.0 255.255.255.0
push "redirect-gateway def1"
push "dhcp-option DNS 8.8.8.8"
ifconfig-pool-persist ipp.txt
keepalive 10 120
cipher BF-CBC        # Blowfish (default)
comp-lzo
#max-clients 10
user nobody
group nobody
push "dhcp-option DNS 8.8.8.8"   
persist-key
persist-tun
status /var/log/openvpn.log
verb 4
mute 10

I have used an absolute minimum set of pf rules based on the discussion I found at Thread 44048.

My pf.conf is:
Code:
vpn_net  = "10.8.0.0/24"
ext_if   = "em0"
# --- NAT  rules -------------
nat on $ext_if from $vpn_net to any -> $ext_if

I can connect to the server via OpenVPN with no problem but I can not ping or get to Internet beyond that

Any suggestions would be appreciated.
 
You need to enable routing, add to /etc/rc.conf:
Code:
gateway_enable="YES"
 
Sorry I should have stated that It is Here is my rc.conf

Code:
ifconfig_em0="SYNCDHCP"
webmin_enable="YES"
pf_enable="YES"
pf_flags=""
pf_rules="/etc/pf.conf
pflog_enable="YES"
pflog_logfile="/var/log/pflog"
pflog_flags=""
gateway_enable="YES"
apache22_enable=YES"
mysql_enable="YES"
openvpn_if="tun"
openvpn_enable="YES"
openvpn_configfile="/usr/local/etc/openvpn/server.conf"

My sysctl.conf

Code:
net.inet.ip.forwarding=1
 
OK I have been googling and the like and tweaked my pf.conf to

Code:
vpn_net  = "172.16.9.0/24"
ext_if   = "em0"
vpn_if   = "tun0"
nat on $ext_if from $vpn_net to any -> ($ext_if)
pass in quick proto tcp from any to port 1194 keep state label "openvpn"
pass quick on $vpn_if keep state


I changed by server.conf to prevent overlaps

Code:
port 1194
proto tcp
dev tun
ca /usr/local/etc/ssl-admin/active/ca.crt
cert /usr/local/etc/ssl-admin/active/server.crt
key /usr/local/etc/ssl-admin/active//server.key
dh /usr/local/etc/ssl-admin/dh2048.pem
server  172.16.9.0 255.255.255.0
push "redirect-gateway def1"
push "dhcp-option DNS 8.8.8.8"
ifconfig-pool-persist ipp.txt
keepalive 10 120
cipher BF-CBC        # Blowfish (default)
comp-lzo
user nobody
group nobody
push "dhcp-option DNS 8.8.8.8"   
persist-key
persist-tun
status /var/log/openvpn.log
verb 9
mute 10

Still no bas through traffic I looked at my routes using netstat -nr

With Client not connected to the VPN i get

Code:
Routing tables
Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            10.0.1.1           UGS         0      406    em0
10.0.1.0/24        link#1             U           0      491    em0
10.0.1.24          link#1             UHS         0        0    lo0
127.0.0.1          link#3             UH          0      152    lo0
172.16.9.0/24      172.16.9.2         UGS         0        0   tun0
172.16.9.1         link#4             UHS         0        0    lo0
172.16.9.2         link#4             UH          0        0   tun0

with it Connected to as 172.16.9.5 I get
Code:
Routing tables
Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            10.0.1.1           UGS         0      437    em0
10.0.1.0/24        link#1             U           0      602    em0
10.0.1.24          link#1             UHS         0        0    lo0
127.0.0.1          link#3             UH          0      152    lo0
172.16.9.0/24      172.16.9.2         UGS         0        0   tun0
172.16.9.1         link#4             UHS         0        0    lo0
172.16.9.2         link#4             UH          0        0   tun0

I would think that a route is needed from client (172.16.9.5) to the gateway?
 
I am typing this from an ipod so I could not look all your conf files in detail. I had similar problem on the latest openBSD which turn out to be mis configured OpenVPN server. Check this thread. http://daemonforums.org/showthread.php?t=7953 However FreeBSD uses obsolete version of PF soi could not compare your pf rules to mine.
 
I am testing PF as I was lead to believe IPFW was obsolete
I took a look at the link, and saw your change to the OpenVPN server.conf

It seems logical that I am doing something wring with OpenVPN

On your post the subnet number confused me as you jumped in on a thread invoking other addresses so I wasn't sure what route you were pushing in

Code:
push "route 192.168.2.0 255.255.255.0"
I assumed the VPN net

I tried changing myserver.conf to
Code:
local 10.0.1.24
port 1194
proto tcp
dev tun
ca /usr/local/etc/ssl-admin/active/ca.crt
cert /usr/local/etc/ssl-admin/active/server.crt
key /usr/local/etc/ssl-admin/active//server.key
dh /usr/local/etc/ssl-admin/dh2048.pem
server  172.16.9.0 255.255.255.0
push "redirect-gateway def1"
#push "dhcp-option DNS 8.8.8.8"
push "route 172.16.090 255.255.255.0"
ifconfig-pool-persist ipp.txt
keepalive 10 120
#tls-auth /usr/local/etc/openvpn/keys/ta.key 0 # This file is secret
cipher BF-CBC        # Blowfish (default)
comp-lzo
user nobody
group nobody
push "dhcp-option DNS 8.8.8.8"   
persist-key
persist-tun
status /var/log/openvpn.log
verb 9
mute 10[

it would seem that the push route is redundant I also tried out using
Code:
push "route 10.0.1.1 255.255.255.0"

neither version fixed the issue
 
jasonhirsh said:
I am testing PF as I was lead to believe IPFW was obsolete
I took a look at the link, and saw your change to the OpenVPN server.conf

The version of PF which FreeBSD is really old comparing to what we use in OpenBSD so most OpenBSD guys would not even bother to look your PF rules. PF unlike OpenSSH or OpenSMTPd server has never meant to be portable and relies heavily on OpenBSD network stack which is as you know completely different OS after more than 20 years separate development from OpenBSD.

jasonhirsh said:
It seems logical that I am doing something wring with OpenVPN

On your post the subnet number confused me as you jumped in on a thread invoking other addresses so I wasn't sure what route you were pushing in

Code:
push "route 192.168.2.0 255.255.255.0"

My VPN network is 10.10.0.0 and 192.168.2.0 is my private network behind OpenVPN server which I am trying to reach from VPN clients.
 
OK I am now confused I am trying to access the internet through the VPN tunnel You said "Access Private network behind VPNi tried

My VPN Network is 172.16.9.0

My client is on 10.0.1.0
I have trued both network addresses in the push with no internet connectivity

From the discussion in this forum it would appear that this is feasible with PF obsolete or not on FreeBSD

I do appreciate you trying to help across OS
 
Ok. Clarification. My VPN clients have physical address 128.xxx.xxx.xxx (it is a large corporate sub-net). My goal is to use my OpenVPN gateway to allow them to connect to my internal sub-net 192.168.2.0/24 so that they can access my DNS server, LDAP server, NFS shares on our File server and few other services which I available only on my internal sub-net 192.168.2.0/24. Note by default OpenVPN clients are not aware of each other (please check the default OpenVPN server configuration file) and I am keeping it that way.

OpenVPN server acts as dhcp leasing server and each VPN client is assigned vpn address on tun0 interface 10.10.0.0/24. In particular OpenVPN server external interface em0 besides IP address 72.23.174.17 has also virtual tun0 interface with address 10.10.0.1.
In order to reach 192.168.2.0/24 I am

Code:
push "route 192.168.2.0 255.255.255.0"

On my OpenVPN server.

My relevant PF rules on OpenVPN server which acts as a gateway into internal network look something like.

Code:
# macros
ext_if="em0"
int_if="em1"
vpn_if="tun0"
lan = "{192.168.2.0/24}"
vpn = "{10.10.0.0/24}"
icmp_types="echoreq"

tcp_services_ext = "{ssh, http, https, submission}"
tcp_services_int = "{ssh, http, https, submission, ldap, \
        111, 2049, 4000, 4001, 4002}"
udp_services = "{domain, ntp}"
udp_services_int = "{domain, ntp, 111, 2049, 4000, 4001, 4002}"
openvpn = "{1194}"

# options
set limit states 100000
set block-policy return
set optimization normal
set loginterface egress
set skip on lo

# match rules
match in all scrub (no-df max-mss 1440)
match out all scrub (no-df max-mss 1440)
match out on egress inet from !(egress:network) to any nat-to (egress:0)

# filter rules
block all
antispoof quick for { lo $int_if }
block drop in quick from urpf-failed to any
pass in log on egress inet proto udp to (egress) port $openvpn

pass on $vpn_if inet proto tcp from $auton_vpn to any port $tcp_services_int
pass on $vpn_if inet proto udp from $auton_vpn to any port $udp_services_int

Now could you please describe one more time your network topology and what you are trying to accomplish?
 
OK lets start with what I am trying to achieve. I am trying to a OpenVPN server on my leased stand alone server. after several major issues it was suggested that I try to build a testbed using VirtualBox and work out the firewall and VPN settings there I have installed FreeeBSD 9.2 on a Virtual Server on one of my Mac OS X boxes and am ruining PF The network it installed on is 10.0.1.0 with gateway at router on 10.0.1.1. The router has a physical internet address of a.b.c.d The Virtual Box Network configuration has bridged from 10.0.1.24 to the ethernet adapter on the Mac OS X box (en0). (that box has an IP of 10.0.1.100)

I have configured the router so that I an access the Virtual Server from a remote server so that i can test from a different physical location
So what I am trying to do is connect to the virtual server from either remotely or from the local box over a VPN and be able to get to the
internet using its connection

_________________ Remote (w.x.y.z)

|
|

Internet__________________ Virtual server (10.0.1.24 (em0)
a.b.c.d (VPN range 172.16.9.0)
(forwarded) |
|
|
c ______________________ Local (10.0.1.100 (en0)



The OpenVPM server on the Virtual server is assigning ups to the OpenVPN clients in the 172.16.9.0

What are the results of ifconfig?
 
I will admit I had really hard time following your last post. It looks like you are trying to set up VPN server on the VirtualBox instance in such a fashion that the connection from remote host as well as VirtualBox is become encrypted? I do know little bit about VirtualBox. So first thing first.

VirtualBox instance should be in bridged mode (which looks like it is). In that case VirtualBox instance which uses the same physical interface as VirtualBox host will be assigned a separate sud-physical IP address. Who is giving you that address and is it dynamic or static? For example if your
VirtualBox host has address

128.0.0.1
your
VirtualBox interface might have address

128.0.0.58

From the ISP point of view 128.0.0.58 has no connection to 128.0.0.1 except that they are on the same subnet. Once in bridged mode virutal box instance really doesn't care if you are trying to connect from the same machine running VirtualBox or from machine 5000 miles away.

VPN server running on VirtualBox instance should listen on 128.0.0.58 and will assign an IP address to tun0 which is on the same "physical interface" as 128.0.0.58. Lets call it 10.0.0.1. Once OpenVPN connection is established from a client wherever it might be that client is going to get address
10.0.0.XXX

Am I following so far what you described above? I am just curios what is the purpose of such set up? The only thing I can think off is that you are running in VirtualBox instance some kind service let say HTTP server and you want only people with VPN credentials to be able to see it (even that kind access control probably could be done easier without OpenVPN and you could still encrypt the traffic with SSL).
 
VirtualBox instance should be in bridged mode (which looks like it is). In that case VirtualBox instance which uses the same physical interface as VirtualBox host will be assigned a separate sud-physical IP address. Who is giving you that address and is it dynamic or static? For example if your
VirtualBox host has address

128.0.0.1
your
VirtualBox interface might have address

128.0.0.58

My virtual box host box has an IP address of 10.0.1.100 In constructing the bridge an IP of 10.0.1.24 has been assigned to the virtual interface (em0)

VPN server running on VirtualBox instance should listen on 128.0.0.58 and will assign an IP address to tun0 which is on the same "physical interface" as 128.0.0.58. Lets call it 10.0.0.1. Once OpenVPN

I think I follow you.... in my situation i think you are saying. I have changed my server.conf

Code:
local 10.0.1.24
port 1194
proto udp
dev tun
ca /usr/local/etc/ssl-admin/active/ca.crt
cert /usr/local/etc/ssl-admin/active/server.crt
key /usr/local/etc/ssl-admin/active//server.key
dh /usr/local/etc/ssl-admin/dh2048.pem
server 172.16.0.0 255.255.255.0 
push "redirect-gateway def1"
ifconfig-pool-persist ipp.txt
keepalive 10 120
cipher BF-CBC        # Blowfish (default)
comp-lzo
#max-clients 10
user nobody
group nobody
push "dhcp-option DNS 8.8.8.8"   
persist-key
persist-tun
status /var/log/openvpn.log
verb 9
mute 10

but that leaves me with this

Code:
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
	ether 08:00:27:48:82:22
	inet 10.0.1.24 netmask 0xffffff00 broadcast 10.0.1.255
	inet6 fe80::a00:27ff:fe48:8222%em0 prefixlen 64 scopeid 0x1 
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128 
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 
	inet 127.0.0.1 netmask 0xff000000 
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
	options=80000<LINKSTATE>
	inet6 fe80::a00:27ff:fe48:8222%tun0 prefixlen 64 scopeid 0x4 
	inet 172.16.0.1 --> 172.16.0.2 netmask 0xffffffff 
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	Opened by PID 713


Am I following so far what you described above? I am just curios what is the purpose of such set up? The only thing I can think off is that you are running in VirtualBox instance some kind service let say HTTP server and you want only people with VPN credentials to be able to see it (even that kind access control probably could be done easier without OpenVPN and you could still encrypt the traffic with SSL).




The purpose of the set up is to test configuration of production server. I am trying to seventually et up that server to host an OpenVPN server so that I can contact with a client and route all internet traffic through that server. In working on that and developing a NAT through IPFW then eventually PF I learned it was best to experiment on a test server. This is what I am doing now Yes I have gotten a SSH tunnel to forward just HTTP traffic.
 
jasonhirsh said:
but that leaves me with this

Code:
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
	ether 08:00:27:48:82:22
	inet 10.0.1.24 netmask 0xffffff00 broadcast 10.0.1.255
	inet6 fe80::a00:27ff:fe48:8222%em0 prefixlen 64 scopeid 0x1 
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128 
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 
	inet 127.0.0.1 netmask 0xff000000 
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
	options=80000<LINKSTATE>
	inet6 fe80::a00:27ff:fe48:8222%tun0 prefixlen 64 scopeid 0x4 
	inet 172.16.0.1 --> 172.16.0.2 netmask 0xffffffff 
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	Opened by PID 713

That is to be expected. The interface tun0 has to be turned on. On OpenBSD that is done by editing /etc/hostname.tun0 file and putting

Code:
up
!/usr/local/sbin/openvpn --daemon --config /etc/openvpn/predrag-server.conf
and rebooting OS which runs in VirtualBox or restarting network scripts.

jasonhirsh said:
The purpose of the set up is to test configuration of production server. I am trying to seventually et up that server to host an OpenVPN server so that I can contact with a client and route all internet traffic through that server. In working on that and developing a NAT through IPFW then eventually PF I learned it was best to experiment on a test server. This is what I am doing now Yes I have gotten a SSH tunnel to forward just HTTP traffic.
Unless OpenVPN server is going to run the VirtualBox you just wasted hole day trouble shooting VirtualBox instead of real hardware. Virtualization except in some very particular instances is one of the worst ideas IMHO that become popular in IT world. Do yourself a favor and get two cheap laptops and forget about VirtualBox and similar crap.
 
Well Guess i am at the end of the line then No such file in FreeBSD

I can see your point on Visualization . Very valid.. Not really a waste of a day. I have been attacking this for over a year
Sorry for wasting your time
 
jasonhirsh said:
Well Guess i am at the end of the line then No such file in FreeBSD
Of course not. FreeBSD has a different network stack. It probably something similar which you put into /etc/rc.conf. Sorry I use FreeBSD only on file servers so I would have to check my notes to be sure. I thought we were on FreeBSD forum that is very elementary thing.
jasonhirsh said:
I have been attacking this for over a year
Sorry for wasting your time
No need to apologize. I can belive what you are saying "Over a year". Are you serious? Once I started playing with it (reading all man pages and great OpenVPN documentation) I had working prototype within three days. Deployment at my work took 2 hours from the moment I installed OpenBSD on our OpenVPN gateway to the moment we had 2 dozen Linux clients connected to it. OpenVPN is production grade software and so is FreeBSD. This is a routine exercise for entry level system admin.
 
I was attacking both firewall and OpenVPN ... I have had OpenVPN working for the 7 years or so that I have had the production server just not the ability to pass through That is the thing that I keep attacking It seems to be striking a balance between the firewall NAT and OpenVPN
 
Fix your nat rule to just nat anything not from the WAN interface to the primary IP ( the :0 added on) of the external interface and have a pass out rule to allow outbound traffic.

Code:
ext_if = "em0"
nat on $ext_if from !($ext_if) to any -> ($ext_if:0)
pass in quick proto tcp from any to port 1194 keep state label "openvpn"
pass out
 
Forgot one thing, make sure you are allowing traffic inbound on the tunnel. This would be the bare minimum you would need to get by. This would only allow inbound VPN connections on the wired interface and allow any traffic from a VPN client. By virtue of making a state entry, the pass out would make sure anything leaving can come back.
Code:
ext_if = "em0"
vpn_if = "tun0"
nat on $ext_if from !($ext_if) to any -> ($ext_if:0)
pass in quick proto tcp from any to port 1194 keep state label "openvpn"
pass in quick on $vpn_if
pass out

From there, work your way from the closest hop outwards. Your earlier netstat -nr didn't reflect your clients getting any kind of route from the server. At that point, you should have had success only running ping 172.16.0.1. You would have gotten further after adding the following line.
Code:
push "redirect-gateway def1"

Based off your IPs, please let me know where things break when you do the following from your connected VPN client:
ping 172.16.9.5 (sanity check, make sure client can ping itself)
ping 172.16.0.1 (shows VPN connectivity--this should have always worked)
ping 10.0.1.24 (shows VPN connectivity and routing configuration took but leaves PF/gateway configuration out)
ping 10.0.1.100 (shows PF/gateway configuration works)
 
The version of PF which FreeBSD is really old comparing to what we use in OpenBSD so most OpenBSD guys would not even bother to look your PF rules. PF unlike OpenSSH or OpenSMTPd server has never meant to be portable and relies heavily on OpenBSD network stack which is as you know completely different OS after more than 20 years separate development from OpenBSD.



My VPN network is 10.10.0.0 and 192.168.2.0 is my private network behind OpenVPN server which I am trying to reach from VPN clients.
Hey, Oko. Could you please give me a hint for my (similar) problem, seeing that I run OpenBSD and PF on my office OpenVPN server with the configuration similar to yours: my VPN network is 10.0.0.0/24 and I push route 192.168.0.0/27 to my clients.

Clients connect OK and they have the route to 192.168.0.0/27 just fine. The problem is I cannot ping 10.0.0.1 (server itself) from the server itself (no route to host)! Nor the client 10.0.0.9 for that matter, even though route show shows a route to 10.0.0/24 via 10.0.0.1.

Here's my /etc/pf.conf:
Code:
all_if = "{ rl0 rl1 rl2 }"
anchor "ftp-proxy/*"
anchor openvpn

set block-policy return
set skip on lo0

match in all scrub (no-df)
match out all scrub (no-df)
match out on egress inet from !egress:network to any nat-to egress:0
match out on rl0 inet from !rl0:network to any nat-to rl0:0
match out on rl2 inet from !rl2:network to any nat-to rl2:0

pass in quick on rl1 inet proto udp from any to any port = 22345
pass in quick on rl1 inet proto tcp to any port ftp divert-to 127.0.0.1 port 8021
block in quick on rl1 from any to any

pass out quick on $all_if modulate state

pass in quick on !egress from any to any modulate state
block in quick all
block out quick all
(here rl1 is the outbound interface, others are local)

And here's my /etc/pf.conf.openvpn for anchor openvpn in /etc/pf.conf:
Code:
pass out quick on tun0 from any to 10.0.0.0/24 modulate state nat-to tun0:0 round-robin
pass in quick on tun0 from any to any modulate state
which is loaded in /etc/hostname.tun0, second line after activating openvpn -- so that I could deactivate openvpn anytime without affecting pf.rules. This works OK, according to the output from pfctl -a openvpn -sr.

Perhaps, I have some misconception with how "anchor" works? I assume it inserts its rules right where the "anchor" directive is placed in pf.conf file? Well, otherwise I would have no connectivity at all.

So why could it be the the server doesn't see the client at all, nor even its own 10.0.0.1 interface? Maybe I don't need to nat-to tun0:0 at all? Perhaps, it's done automatically? I've tried a lot of variations and just have no idea now.

Thank you very much in advance.
 
Maybe I don't need to nat-to tun0:0 at all?
Yes, this is correct. I have left this:
Code:
pass out quick on tun0 from any to any modulate state
pass in quick on tun0 from any to any modulate state
and now the issue is fixed. Tomorrow I will check if I can ping from the 192.168.0.0/27 office network to a connected VPN client (10.0.0.9).

BTW, it's OpenBSD 5.5 with all the latest stuff.
 
Back
Top