I know, this appears to be answered all over the place. But none of the answers are working for me so I must have misunderstood something somewhere along the way.
Background: I'm trying to set up a FreeBSD jail that acts as a router. Inside that jail, I'd like to run OpenVPN. I would like all traffic I send to the FreeBSD jail to route through the OpenVPN connection and not through any other interface.
I'm running FreeBSD 11.2-RELEASE-p2. I've installed iocage to assist with managing the jail environments. After creating a new jail, I can console into the jail and touch the internet. The settings for the jail are:
/etc/defaults/devfs.rules
Log in to the console.
Check to see it works.
What networks do I have inside the jail?
So far so good. However, if I set my default route (on another host: 192.168.11.3) to 192.168.11.5, no traffic routes from that host. Trying to ping google from 192.168.11.3 with 192.168.11.5 as the default route. tcpdump'ing from the host shows:
The host has:
I can't set that on the jail.
At this point, what am I doing wrong? Most of the places that detail how to fix this type of issue show something like cloning the loopback and then utilizing pf to perform NAT. I could probably copy that, but I am trying to understand what I'm doing wrong here.
Of course, the VPN doesn't work either. I can start the vpn but it doesn't stay connected. It fails as it cannot set the IP on the tun interface. I will ask more about this later, but I wanted to understand my issue with normal routing first. But for those that are easily able to understand and fix this I'll explain.
On the host I create a tun interface.
Both the host and the jail see tun0. However, when starting OpenVPN the following error happens.
Thanks in advance for any help you can offer.
Background: I'm trying to set up a FreeBSD jail that acts as a router. Inside that jail, I'd like to run OpenVPN. I would like all traffic I send to the FreeBSD jail to route through the OpenVPN connection and not through any other interface.
I'm running FreeBSD 11.2-RELEASE-p2. I've installed iocage to assist with managing the jail environments. After creating a new jail, I can console into the jail and touch the internet. The settings for the jail are:
Code:
{
"CONFIG_VERSION": "9",
"allow_chflags": "0",
"allow_mount": "0",
"allow_mount_devfs": "0",
"allow_mount_nullfs": "0",
"allow_mount_procfs": "0",
"allow_mount_tmpfs": "0",
"allow_mount_zfs": "0",
"allow_quotas": "0",
"allow_raw_sockets": "1",
"allow_set_hostname": "1",
"allow_socket_af": "1",
"allow_sysvipc": "1",
"available": "readonly",
"basejail": "no",
"boot": "off",
"bpf": "no",
"children_max": "0",
"cloned_release": "11.2-RELEASE",
"comment": "none",
"compression": "lz4",
"compressratio": "readonly",
"coredumpsize": "off",
"count": "1",
"cpuset": "off",
"cputime": "off",
"datasize": "off",
"dedup": "off",
"defaultrouter": "192.168.11.1",
"defaultrouter6": "none",
"depends": "none",
"devfs_ruleset": "4",
"dhcp": "off",
"enforce_statfs": "2",
"exec_clean": "1",
"exec_fib": "0",
"exec_jail_user": "root",
"exec_poststart": "/usr/bin/true",
"exec_poststop": "/usr/bin/true",
"exec_prestart": "/usr/bin/true",
"exec_prestop": "/usr/bin/true",
"exec_start": "/bin/sh /etc/rc",
"exec_stop": "/bin/sh /etc/rc.shutdown",
"exec_system_jail_user": "0",
"exec_system_user": "root",
"exec_timeout": "60",
"host_domainname": "none",
"host_hostname": "vpn_router",
"host_hostuuid": "vpn_router",
"host_time": "yes",
"hostid": "00000000-0000-0000-0000-002590f006d6",
"interfaces": "none",
"ip4": "inherit",
"ip4_addr": "igb0|192.168.11.5/24",
"ip4_saddrsel": "1",
"ip6": "new",
"ip6_addr": "none",
"ip6_saddrsel": "1",
"jail_zfs": "off",
"jail_zfs_dataset": "iocage/jails/vpn_router/data",
"jail_zfs_mountpoint": "none",
"last_started": "2018-09-08 13:32:34",
"login_flags": "-f root",
"mac_prefix": "02ff60",
"maxproc": "off",
"memorylocked": "off",
"memoryuse": "off",
"mount_devfs": "1",
"mount_fdescfs": "1",
"mount_linprocfs": "0",
"mount_procfs": "0",
"mountpoint": "readonly",
"msgqqueued": "off",
"msgqsize": "off",
"nmsgq": "off",
"notes": "none",
"nsemop": "off",
"nshm": "off",
"nthr": "off",
"openfiles": "off",
"origin": "readonly",
"owner": "root",
"pcpu": "off",
"priority": "99",
"pseudoterminals": "off",
"quota": "none",
"release": "11.2-RELEASE-p2",
"reservation": "none",
"resolver": "/etc/resolv.conf",
"rlimits": "off",
"securelevel": "2",
"shmsize": "off",
"stacksize": "off",
"stop_timeout": "30",
"swapuse": "off",
"sync_state": "none",
"sync_target": "none",
"sync_tgt_zpool": "none",
"sysvmsg": "new",
"sysvsem": "new",
"sysvshm": "new",
"template": "no",
"type": "jail",
"used": "readonly",
"vmemoryuse": "off",
"vnet": "off",
"vnet0_mac": "none",
"vnet1_mac": "none",
"vnet2_mac": "none",
"vnet3_mac": "none",
"wallclock": "off"
}
/etc/defaults/devfs.rules
Code:
[devfsrules_jail=4]
add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add path 'tun*' unhide
add path zfs unhide
Code:
iocage start vpn_router
* Starting vpn_router
+ Started OK
+ Starting services OK
Log in to the console.
Code:
iocage console vpn_router
Last login: Sat Sep 8 09:04:09 on pts/2
FreeBSD 11.2-RELEASE-p2 (GENERIC) #0:
Check to see it works.
Code:
fetch -qo - http://ifconfig.co
***.***.***.200
What networks do I have inside the jail?
Code:
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:25:90:f0:06:d6
hwaddr 00:25:90:f0:06:d6
inet 192.168.11.5 netmask 0xffffff00 broadcast 192.168.11.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
So far so good. However, if I set my default route (on another host: 192.168.11.3) to 192.168.11.5, no traffic routes from that host. Trying to ping google from 192.168.11.3 with 192.168.11.5 as the default route. tcpdump'ing from the host shows:
Code:
tcpdump -i igb0 host 192.168.11.5
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on igb0, link-type EN10MB (Ethernet), capture size 262144 bytes
09:23:12.083933 ARP, Request who-has 192.168.11.5 tell 192.168.11.3, length 46
09:23:12.083965 ARP, Reply 192.168.11.5 is-at 00:25:90:f0:06:d6 (oui Unknown), length 28
09:23:12.112283 ARP, Request who-has 192.168.11.5 tell 192.168.11.3, length 46
09:23:12.112304 ARP, Reply 192.168.11.5 is-at 00:25:90:f0:06:d6 (oui Unknown), length 28
09:23:12.114072 IP 192.168.11.3.61798 > 192.168.11.5.domain: 35187+ PTR? db._dns-sd._udp.0.11.168.192.in-addr.arpa. (59)
09:23:12.114078 IP 192.168.11.3.55078 > 192.168.11.5.domain: 64239+ PTR? b._dns-sd._udp.0.11.168.192.in-addr.arpa. (58)
09:23:12.114098 IP 192.168.11.5 > 192.168.11.3: ICMP 192.168.11.5 udp port domain unreachable, length 36
09:23:12.114100 IP 192.168.11.5 > 192.168.11.3: ICMP 192.168.11.5 udp port domain unreachable, length 36
The host has:
Code:
gateway_enable="YES"
cloned_interfaces="tun"
I can't set that on the jail.
Code:
root@vpn_router:~ # sysctl net.inet.ip.forwarding=1
net.inet.ip.forwarding: 1
sysctl: net.inet.ip.forwarding=1: Operation not permitted
At this point, what am I doing wrong? Most of the places that detail how to fix this type of issue show something like cloning the loopback and then utilizing pf to perform NAT. I could probably copy that, but I am trying to understand what I'm doing wrong here.
Of course, the VPN doesn't work either. I can start the vpn but it doesn't stay connected. It fails as it cannot set the IP on the tun interface. I will ask more about this later, but I wanted to understand my issue with normal routing first. But for those that are easily able to understand and fix this I'll explain.
On the host I create a tun interface.
Code:
ifconfig tun0 create
Both the host and the jail see tun0. However, when starting OpenVPN the following error happens.
Code:
Sat Sep 8 09:39:02 2018 us=495288 TUN/TAP device /dev/tun0 opened
Sat Sep 8 09:39:02 2018 us=495308 do_ifconfig, tt->did_ifconfig_ipv6_setup=1
Sat Sep 8 09:39:02 2018 us=495360 /sbin/ifconfig tun0 10.***.***.*** 10.***.***.1 mtu 1500 netmask 255.255.255.0 up
ifconfig: ioctl SIOCSIFMTU (set mtu): Operation not permitted
Sat Sep 8 09:39:02 2018 us=498640 FreeBSD ifconfig failed: external program exited with error status: 1
Sat Sep 8 09:39:02 2018 us=498694 Exiting due to fatal error