Hi,
I have OpenVPN in a jail on one machine and natively on another. Both machines are VMs running FreeBSD 13. I am trying to redirect a port from one machine to the other without losing the source address. This works, as the packets easily traverse where they need to go, as verified with tcpdump on BOTH ENDS. It is well thought out (I think) with a second routing table and a separate IP in addition to the one assigned by VirtualBox DHCP to make sure that packets end up on correct devices. And they do. However, the client unexpectly breaks up the connection with a TCP RST. The setup and details look the following:
CLIENT
/boot/loader.conf
/etc/pf.conf
crontab (root)
/etc/jail.conf
JAIL 1 - ovpn-cli - (ON THE CLIENT)
/usr/local/etc/openvn/openvpn.conf
Starting manually from the host:
JAIL 2 - mail - (ON THE SERVER)
Jail is started manually on the host:
SERVER
ifconfig
/usr/local/etc/openvpn/openvpn.conf
/etc/pf.conf
I start OpenVPN manually:
Connectivity works, e.g. ssh 10.8.0.2 from the server or ssh 10.8.0.1 from the client. All perfect
Then I try to connect to 10.0.3.15:2525 which should redirect me to 10.8.0.2:2525 like this:
# telnet 10.0.3.15 2525
I have verified with tcpdump on BOTH ends that the following exchange takes place:
The setup is on two VirtualBox instances and port 1197 on the server is redirected so that 10.0.2.2 (IP of the host PC as visible from the guest) at port 1197 redirects to the guest port 1197.
I did ifconfig -rxcsum -txcsum on lo0 and em0 on both VMs as otherwise I was getting the famous checksum errors with this redirect.
Is there a reason why the client suddenly replies with TCP RST? Could someone who is experienced share an opinion please? Thank you in advance.
Kind regards,
--
Stanislaw
I have OpenVPN in a jail on one machine and natively on another. Both machines are VMs running FreeBSD 13. I am trying to redirect a port from one machine to the other without losing the source address. This works, as the packets easily traverse where they need to go, as verified with tcpdump on BOTH ENDS. It is well thought out (I think) with a second routing table and a separate IP in addition to the one assigned by VirtualBox DHCP to make sure that packets end up on correct devices. And they do. However, the client unexpectly breaks up the connection with a TCP RST. The setup and details look the following:
CLIENT
ifconfig
Code:
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=98<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
ether 08:00:27:76:06:22
hwaddr 08:00:27:76:06:22
inet 10.0.2.15 netmask 0xffffff00 broadcast 10.0.2.255
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600000<RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
groups: lo
lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet 127.0.1.0 netmask 0xffffffff
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
groups: lo
tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
inet 10.8.0.2 --> 10.8.0.1 netmask 0xff000000
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
groups: tun
netstat -rn -F 0
Code:
Routing tables
Internet:
Destination Gateway Flags Netif Expire
default 10.0.2.2 UGS em0
10.0.2.0/24 link#1 U em0
10.0.2.15 link#1 UHS lo0
10.8.0.1 link#4 UH tun0
10.8.0.2 link#4 UHS lo0
127.0.0.1 link#2 UH lo0
127.0.1.0 link#3 UH lo1
Internet6:
Destination Gateway Flags Netif Expire
::/96 ::1 UGRS lo0
::1 link#2 UH lo0
::ffff:0.0.0.0/96 ::1 UGRS lo0
fe80::/10 ::1 UGRS lo0
fe80::%lo0/64 link#2 U lo0
fe80::1%lo0 link#2 UHS lo0
ff02::/16 ::1 UGRS lo0
netstat -rn -F 1
Code:
Routing tables (fib: 1)
Internet:
Destination Gateway Flags Netif Expire
default 10.8.0.1 UGS tun0
[URL='http://10.0.2.0/24']10.0.2.0/24[/URL] link#1 U em0
10.8.0.1 link#4 UH tun0
127.0.0.1 link#2 UH lo0
127.0.1.0 link#3 UH lo1
Internet6:
Destination Gateway Flags Netif Expire
::/96 ::1 UGRS lo0
::1 link#2 UH lo0
::ffff:[URL='http://0.0.0.0/96']0.0.0.0/96[/URL] ::1 UGRS lo0
fe80::/10 ::1 UGRS lo0
fe80::%lo0/64 link#2 U lo0
ff02::/16 ::1 UGRS lo0
Code:
kern.geom.label.disk_ident.enable="0"
kern.geom.label.gptid.enable="0"
zfs_load="YES"
net.fibs="2"
Code:
pass on lo0 all
pass on em0 all
pass on tun0 all
Code:
@every_minute if [ "`ifconfig tun0 | grep inet | cut -d' ' -f2`" != "10.8.0.2" ] ; then ifconfig tun0 inet 10.8.0.2 10.8.0.1 && route add default 10.8.0.1 -fib 1; fi >/dev/null 2>&1
Code:
70681eb {
path = '/focker/jails/70681eb';
persist;
interface = 'lo1';
ip4.addr = '127.0.1.0';
mount.devfs;
exec.clean;
host.hostname = 'ovpn-cli';
exec.start = '/bin/sh /etc/rc';
exec.stop = '/bin/sh /etc/rc.shutdown';
exec.prestart = 'cp /etc/resolv.conf /focker/jails/70681eb/etc/resolv.conf';
}
e1aca4b {
path = '/focker/jails/e1aca4b';
persist;
interface = 'tun0';
ip4.addr = 'tun0|10.8.0.2 10.8.0.1';
mount.devfs;
exec.clean;
host.hostname = 'mail';
exec.start = '/bin/sh /etc/rc';
exec.stop = '/bin/sh /etc/rc.shutdown';
exec.fib = 1;
exec.prestart = 'cp /etc/resolv.conf /focker/jails/e1aca4b/etc/resolv.conf && route add default 10.8.0.1 -fib 1';
}
/usr/local/etc/openvn/openvpn.conf
Code:
proto udp4
dev tun0
remote 10.0.2.2
port 1197
ifconfig 10.8.0.2 10.8.0.1
secret /usr/local/etc/openvpn/static.key
cipher aes-256-cbc
ifconfig-noexec
keepalive 10 60
Code:
# service jail start 70681eb
# jexec 70681e
jail # openvpn /usr/local/openvpn/openvpn.conf
Jail is started manually on the host:
Code:
# service jail start e1aca4b
# setfib 1 jexec e1aca4b
jail # nc -l 2525
ifconfig
Code:
em0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4810098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWFILTER,NOMAP>
ether 08:00:27:ed:9b:4f
inet 10.0.2.15 netmask 0xffffff00 broadcast 10.0.2.255
inet 10.0.3.15 netmask 0xffffffff broadcast 10.0.3.15
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8149<UP,LOOPBACK,RUNNING,PROMISC,MULTICAST> metric 0 mtu 16384
options=680000<LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
netstat -rn
Code:
Routing tables
Internet:
Destination Gateway Flags Netif Expire
default 10.0.2.2 UGS em0
[URL='http://10.0.2.0/24']10.0.2.0/24[/URL] link#1 U em0
10.0.2.15 link#1 UHS lo0
10.0.3.15 link#1 UH em0
127.0.0.1 link#2 UH lo0
Internet6:
Destination Gateway Flags Netif Expire
::/96 ::1 UGRS lo0
::1 link#2 UHS lo0
::ffff:[URL='http://0.0.0.0/96']0.0.0.0/96[/URL] ::1 UGRS lo0
fe80::/10 ::1 UGRS lo0
fe80::%lo0/64 link#2 U lo0
fe80::1%lo0 link#2 UHS lo0
ff02::/16 ::1 UGRS lo0
Code:
proto udp4
dev tun0
port 1197
ifconfig 10.8.0.1 10.8.0.2
secret /usr/local/etc/openvpn/static.key
cipher aes-256-cbc
keepalive 10 60
Code:
rdr pass inet proto tcp from any to 10.0.3.15 port 2525 -> 10.8.0.2 port 2525
pass on em0 all
pass on lo0 all
pass on tun0 all
Code:
# openvpn /usr/local/etc/openvpn/openvpn.conf
Then I try to connect to 10.0.3.15:2525 which should redirect me to 10.8.0.2:2525 like this:
# telnet 10.0.3.15 2525
I have verified with tcpdump on BOTH ends that the following exchange takes place:
Code:
12:53:19.632773 IP (tos 0x10, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 48)
10.0.3.15.26360 > 10.8.0.2.2525: Flags , cksum 0x3d15 (correct), seq 2149400320, win 65535, options [mss 1460,sackOK,eol], length 0
12:53:19.636213 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 48)
10.8.0.2.2525 > 10.0.3.15.26360: Flags [S.], cksum 0x6969 (correct), seq 1358332690, ack 2149400321, win 65535, options [mss 1350,sackOK,eol], length 0
12:53:19.636223 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 40)
10.0.3.15.26360 > 10.8.0.2.2525: Flags [R], cksum 0x68d4 (correct), seq 2149400321, win 0, length 0
I did ifconfig -rxcsum -txcsum on lo0 and em0 on both VMs as otherwise I was getting the famous checksum errors with this redirect.
Is there a reason why the client suddenly replies with TCP RST? Could someone who is experienced share an opinion please? Thank you in advance.
Kind regards,
--
Stanislaw