Hello,
I'm facing a very strange problem. Yesterday I've replaced the nginx web server on my home internet gateway with Caddy2 (from pkg). The gateway runs FreeBSD 12.2-RELEASE and does NAT and firewall (pf) for my LAN. This gateway has been running for ages with no particular network problems.
Motivation for using Caddy web server was to learn something new, gain QUIC protocol and LetsEncrypt automation. Fact is Caddy can't open restricted ports as root then drop privileges. I've added this to /boot/loader.conf so that Caddy would run as non-privileged user and could open ports 80 and 443:
And this to /etc/sysctl.conf:
and rebooted.
Caddy was working but I've got warnings in the logs:
Following documentation I've increased kern.ipc.maxsockbuf to 3014656 (then later to 6014656).
Later that day, I've had a network disconnection: PCs on the LAN have lost internet access, etc. I've rebooted the FreeBSD gateway, network was back.
Then we started to watch Netflix (from a device on the LAN), after ~20 minutes of streaming network went down. New reboot, Netflix we go, 20 minutes later network went down again.
I've disabled Caddy in rc.conf, and commented out every modification I've made in /boot/loader.conf and /etc/sysctl.conf, rebooted, and started Netflix again: it went smoothly, no more network downtime…
On the gateway,
em0 being LAN side and em1 Internet side.
Symptoms on the gateway were "evolving": right after the connection went down I would have
Any help greatly appreciated… I might have missed something obvious but I'm pretty sure my config is OK and should work.
Thanks!
I'm facing a very strange problem. Yesterday I've replaced the nginx web server on my home internet gateway with Caddy2 (from pkg). The gateway runs FreeBSD 12.2-RELEASE and does NAT and firewall (pf) for my LAN. This gateway has been running for ages with no particular network problems.
Motivation for using Caddy web server was to learn something new, gain QUIC protocol and LetsEncrypt automation. Fact is Caddy can't open restricted ports as root then drop privileges. I've added this to /boot/loader.conf so that Caddy would run as non-privileged user and could open ports 80 and 443:
Code:
mac_portacl_load="YES"
Code:
security.mac.portacl.port_high=1023
net.inet.ip.portrange.reservedlow=0
net.inet.ip.portrange.reservedhigh=0
security.mac.portacl.suser_exempt=1
security.mac.portacl.rules=uid:1007:tcp:80,uid:1007:tcp:443,uid:1007:udp:80,uid:1007:udp:443Caddy was working but I've got warnings in the logs:
Code:
2022/02/24 17:57:40 failed to increase receive buffer size: set udp [::]:443: setsockopt: no buffer space available. See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details.Following documentation I've increased kern.ipc.maxsockbuf to 3014656 (then later to 6014656).
Later that day, I've had a network disconnection: PCs on the LAN have lost internet access, etc. I've rebooted the FreeBSD gateway, network was back.
Then we started to watch Netflix (from a device on the LAN), after ~20 minutes of streaming network went down. New reboot, Netflix we go, 20 minutes later network went down again.
I've disabled Caddy in rc.conf, and commented out every modification I've made in /boot/loader.conf and /etc/sysctl.conf, rebooted, and started Netflix again: it went smoothly, no more network downtime…
On the gateway,
ifconfig output was exactly the same before and during network outage: no change in interfaces' status. I've found no message in logs that would shed the light on the root cause. Relevant ifconfig output is:
Code:
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=81249b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LRO,WOL_MAGIC,VLAN_HWFILTER>
ether x:x:x:x:71:40
inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=81249b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LRO,WOL_MAGIC,VLAN_HWFILTER>
ether x:x:x:x:71:41
inet a.b.c.20 netmask 0xffffff00 broadcast a.b.c.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>em0 being LAN side and em1 Internet side.
Symptoms on the gateway were "evolving": right after the connection went down I would have
ping: sendto: No buffer space available when trying to ping internet hosts but after few minutes it would turn into ping: sendto: No route to host.Any help greatly appreciated… I might have missed something obvious but I'm pretty sure my config is OK and should work.
Thanks!
