Hello all,
We have a high capacity custom HTTP server working on FreeBSD (currently 9.2-RELEASE amd64 with generic kernel). Some system limits related to network connections and memory are raised for it to work properly. Here are our settings:
/boot/loader.conf:
/etc/sysctl.conf:
Lately we started having problems, our server stopped accepting new connections and we got the "too many open files" error from kernel repeatedly. We checked limits in the system and kern.ipc.numopensockets was way below limit (a little bit over 100k, while the limit, as seen above, is 200k).
We tried to raise the limit, but it didn't help. We managed to solve the problem for now by shortening the time keep-alive connections are kept, so that kern.ipc.numopensockets stays at about 80k. But this is a temporary solution.
I tried to find other limits that might have impact on the number of network connections, but I have been unsuccessful so far. The machine has 8 GB physical RAM and during the "shortage" of sockets there was no shortage of memory. Also
The HTTP server process works with root credentials.
Any hints what to look for will be greatly appreciated.
We have a high capacity custom HTTP server working on FreeBSD (currently 9.2-RELEASE amd64 with generic kernel). Some system limits related to network connections and memory are raised for it to work properly. Here are our settings:
/boot/loader.conf:
Code:
accf_http_load="YES"
accf_data_load="YES"
kern.maxusers = "1024"
kern.ipc.nmbclusters = "100000"
net.inet.tcp.tcbhashsize = "16384"
compat.ia32.maxdsiz = "1610612736"
compat.ia32.maxssiz = "268435456"/etc/sysctl.conf:
Code:
kern.sugid_coredump=1
net.inet.carp.preempt=0
net.inet.carp.log=2
net.inet.tcp.msl=10000
kern.ipc.somaxconn=32768
net.inet.icmp.icmplim=3000
kern.ipc.maxsockets=200000
net.inet.tcp.delayed_ack=1
net.inet.tcp.keepidle=10000
net.inet.tcp.keepintvl=5000
net.inet.tcp.always_keepalive=1
net.inet.tcp.finwait2_timeout=15000
net.inet.tcp.fast_finwait2_recycle=1
net.inet.tcp.maxtcptw=25000
net.inet.tcp.rfc1323=0Lately we started having problems, our server stopped accepting new connections and we got the "too many open files" error from kernel repeatedly. We checked limits in the system and kern.ipc.numopensockets was way below limit (a little bit over 100k, while the limit, as seen above, is 200k).
We tried to raise the limit, but it didn't help. We managed to solve the problem for now by shortening the time keep-alive connections are kept, so that kern.ipc.numopensockets stays at about 80k. But this is a temporary solution.
I tried to find other limits that might have impact on the number of network connections, but I have been unsuccessful so far. The machine has 8 GB physical RAM and during the "shortage" of sockets there was no shortage of memory. Also
netstat -m didn't show shortage in mbufs.The HTTP server process works with root credentials.
Any hints what to look for will be greatly appreciated.
