Context
I have been running FreeBSD on my server PC for a long while now with jails (Iocage).
Since a few months, I moved to a different place that has a router with IPV6 and adjusted my server to use that as well.
After a power outage / upgrade to FreeBSD 13.3-RELEASE-p3, my server has a lot of issues with resolving connections.
Incoming connections work as a charm, but retrieving stuff from (mostly) https://plex.tv is horrible.
Jails
rc.conf
Setting up IPV6 was only working without activating the last 2 lines for some reason...
resolv.conf
pf.conf
*I removed the lines from the jails that are not interesting for this issue.
netstat
netstat -ra (TAKES AGES)
netstat in plex jail:
More
After noticing that Plex could not retrieve metadata, I started searching for solutions for the issue.
After a lot of time, I tried to simply do a wget on the URLs that threw an 408 connection timeout.
Drilling also takes a long while or just throws network errors, weirdly drilling has intervals of 5 second delays:
Above are 4 drills:
I have a second server connected to the same switch + router, that works like a charm.
I probably have some messed up configuration, but hope you can see what I am doing wrong.
Thank you in advance for your time and help!
I have been running FreeBSD on my server PC for a long while now with jails (Iocage).
Since a few months, I moved to a different place that has a router with IPV6 and adjusted my server to use that as well.
After a power outage / upgrade to FreeBSD 13.3-RELEASE-p3, my server has a lot of issues with resolving connections.
Incoming connections work as a charm, but retrieving stuff from (mostly) https://plex.tv is horrible.
Jails
Code:
# iocage list
+-----+-----------+-------+--------------+-----------------------+
| JID | NAME | STATE | RELEASE | IP4 |
+=====+===========+=======+==============+=======================+
| 4 | apache | up | 13.3-RELEASE | 172.16.10.2 |
+-----+-----------+-------+--------------+-----------------------+
| 3 | databases | up | 13.3-RELEASE | 172.16.10.6 |
+-----+-----------+-------+--------------+-----------------------+
| 1 | nginx | up | 13.3-RELEASE | 172.16.10.5 |
+-----+-----------+-------+--------------+-----------------------+
| 6 | plex | up | 13.3-RELEASE | 172.16.10.1,127.0.0.1 |
+-----+-----------+-------+--------------+-----------------------+
| 5 | squid | up | 13.3-RELEASE | 172.16.10.3 |
+-----+-----------+-------+--------------+-----------------------+
| 2 | unifi | up | 13.3-RELEASE | 172.16.10.8 |
+-----+-----------+-------+--------------+-----------------------+
rc.conf
Code:
hostname="virtualalchemy.nl"
ifconfig_em0="DHCP"
sshd_enable="YES"
dumpdev="AUTO"
zfs_enable="YES"
defaultrouter="192.168.0.1"
# ifconfig_em0="inet 192.168.0.137 netmask 255.255.255.0"
ifconfig_em0="DHCP"
ifconfig_em0_ipv6="inet6 accept_rtadv"
ezjail_enable="NO"
cloned_interfaces="lo1"
ipv4_addrs_lo1="172.16.10.0-9/29"
iocage_enable="YES"
pf_enable="YES"
smartd_enable="YES"
smartd_enable="YES"
# ipv6_enable="YES"
# ipv6_activate_all_interfaces="YES"
Setting up IPV6 was only working without activating the last 2 lines for some reason...
resolv.conf
Code:
# cat /etc/resolv.conf
# Generated by resolvconf
search home
nameserver 1.0.0.1
nameserver 1.1.1.1
nameserver 2606:4700:4700::1001
nameserver 2606:4700:4700::1111
Code:
# ifconfig
em0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4e524bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
ether d0:50:99:9b:27:27
inet6 fe80::d250:99ff:fe9b:2727%em0 prefixlen 64 scopeid 0x1
inet6 fd4c:4b32:3330:0:d250:99ff:fe9b:2727 prefixlen 64 autoconf
inet6 2a02:a456:204c:0:d250:99ff:fe9b:2727 prefixlen 64 autoconf
inet 192.168.0.123 netmask 0xffffff00 broadcast 192.168.0.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet 172.16.10.0 netmask 0xfffffff8
inet 172.16.10.4 netmask 0xffffffff
inet 172.16.10.7 netmask 0xffffffff
inet 172.16.10.9 netmask 0xffffffff
inet 172.16.10.5 netmask 0xffffffff
inet 172.16.10.8 netmask 0xffffffff
inet 172.16.10.6 netmask 0xffffffff
inet 172.16.10.2 netmask 0xffffffff
inet 172.16.10.3 netmask 0xffffffff
inet 172.16.10.1 netmask 0xffffffff
inet6 fe80::1%lo1 prefixlen 64 scopeid 0x3
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pf.conf
Code:
# cat /etc/pf.conf
# Public IP address
IP_PUB="192.168.0.123"
TCP_PASS = "{ 22 }"
# Packet normalization
scrub in all
# Allow outbound connections from within the jails
nat on em0 from lo1:network to any -> (em0)
# Plex + qbittorent
rdr on em0 proto tcp from any to $IP_PUB port 32400 -> 172.16.10.1
rdr on em0 proto tcp from any to $IP_PUB port 8181 -> 172.16.10.1
rdr on em0 proto tcp from any to $IP_PUB port 56000 -> 172.16.10.1
rdr on em0 proto tcp from any to $IP_PUB port 25025 -> 172.16.10.1
# Allow ssh
pass out proto tcp to port $TCP_PASS keep state
*I removed the lines from the jails that are not interesting for this issue.
netstat
netstat -ra (TAKES AGES)
Code:
Routing tables
Internet:
Destination Gateway Flags Netif Expire
default 192.168.0.254 UGS em0
default 192.168.0.1 UGS em0
localhost link#2 UH lo0
172.16.10.0 link#3 UH lo1
172.16.10.1 link#3 UH lo1
172.16.10.2 link#3 UH lo1
172.16.10.3 link#3 UH lo1
172.16.10.4 link#3 UH lo1
172.16.10.5 link#3 UH lo1
172.16.10.6 link#3 UH lo1
172.16.10.7 link#3 UH lo1
172.16.10.8 link#3 UH lo1
172.16.10.9 link#3 UH lo1
192.168.0.0/24 link#1 U em0
192.168.0.123 link#1 UHS lo0
Internet6:
Destination Gateway Flags Netif Expire
::/96 localhost URS lo0
default fe80::ce58:30ff:fe UG em0
localhost link#2 UHS lo0
::ffff:0.0.0.0/96 localhost URS lo0
2a02:a456:204c::/6 link#1 U em0
2a02-a456-204c-0-d link#1 UHS lo0
fd4c:4b32:3330::/6 link#1 U em0
fd4c:4b32:3330:0:d link#1 UHS lo0
fe80::/10 localhost URS lo0
fe80::%em0/64 link#1 U em0
fe80::d250:99ff:fe link#1 UHS lo0
fe80::%lo0/64 link#2 U lo0
fe80::1%lo0 link#2 UHS lo0
fe80::%lo1/64 link#3 U lo1
fe80::1%lo1 link#3 UHS lo0
ff02::/16 localhost URS lo0
netstat in plex jail:
Code:
root@plex:~ # netstat -ra
Routing tables
Internet:
Destination Gateway Flags Netif Expire
localhost link#2 UH lo0
172.16.10.1 link#3 UH lo1
More
After noticing that Plex could not retrieve metadata, I started searching for solutions for the issue.
After a lot of time, I tried to simply do a wget on the URLs that threw an 408 connection timeout.
Code:
root@plex:~ # wget [URL]https://plex.tv[/URL]
--2024-08-13 20:24:28-- [URL]https://plex.tv/[/URL]
Resolving plex.tv (plex.tv)... 52.18.158.99, 54.154.120.177
Connecting to plex.tv (plex.tv)|52.18.158.99|:443... failed: Operation timed out.
Connecting to plex.tv (plex.tv)|54.154.120.177|:443...
Drilling also takes a long while or just throws network errors, weirdly drilling has intervals of 5 second delays:
Code:
root@plex:~ # drill @1.1.1.1 plex.tv
Error: error sending query: Could not send or receive, because of network error
root@plex:~ # drill @1.1.1.1 plex.tv
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 9724
;; flags: qr rd ra ; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; plex.tv. IN A
;; ANSWER SECTION:
plex.tv. 33 IN A 54.154.120.177
plex.tv. 33 IN A 52.18.158.99
;; AUTHORITY SECTION:
;; ADDITIONAL SECTION:
;; Query time: 5051 msec
;; SERVER: 1.1.1.1
;; WHEN: Tue Aug 13 20:26:54 2024
;; MSG SIZE rcvd: 57
root@plex:~ # drill @1.1.1.1 plex.tv
Error: error sending query: Could not send or receive, because of network error
root@plex:~ # drill @1.1.1.1 plex.tv
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 33672
;; flags: qr rd ra ; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; plex.tv. IN A
;; ANSWER SECTION:
plex.tv. 18 IN A 52.18.158.99
plex.tv. 18 IN A 54.154.120.177
;; AUTHORITY SECTION:
;; ADDITIONAL SECTION:
;; Query time: 5 msec
;; SERVER: 1.1.1.1
;; WHEN: Tue Aug 13 20:27:40 2024
;; MSG SIZE rcvd: 57
- fail
- 5 + something seconds
- fail
- directly
I have a second server connected to the same switch + router, that works like a charm.
I probably have some messed up configuration, but hope you can see what I am doing wrong.
Thank you in advance for your time and help!