Too many open files

martinrame

martinrame

In a FreeBSD 13.2 system there's a mounted directory from a Windows server using smbfs, this directory has lots of files distributed in many subdirectories.

In this machine also there's a Jail with the smbfs directory mounted. Then from the jail a Python script opens every file from that directory, extracts some data and save that data to a db, then closes the file, this process is single threaded and should work in background for about one week.

The problem I found is after a while, let's say 10 minutes of running the system emits "Too many open files" error and no more files can be processed until the jail is restarted.

I've read that the files open could be clashing against kern.maxfiles, but its value is 20000 while kern.openfiles is never above ~600.

I'm lost, does anyone know where to look to start debugging this?
 
Anything useful in these?

forums.freebsd.org

Elasticsearch 7 and FreeBSD 12.1 openfiles limit (Jail)

I am trying to install and set up elasticsearch 7 in an iocage jail. I get an error when trying to run elasticsearch: [1]: max file descriptors [16384] for elasticsearch process is too low, increase to at least [65535] I have tried to increase the max file limit of the jail to 65536 by doing...
forums.freebsd.org forums.freebsd.org

Scroll down to "FreeBSD Jails and rctl" - it talks about checking current limits and changing:
klarasystems.com

Controlling Resource Limits with rctl in FreeBSD

FreeBSD provides several methods to limit the system resources an individual may use. The rctl command can be effectively used for controlling resource limits.
klarasystems.com klarasystems.com
 
I think the problem comes from Windows and/or mount_smbfs. So far, now there's almost 50% of the files processed, in a couple more days this will be done. So, I'll keep restarting the jail and let the process finish.
 
martinrame said:
I've read that the files open could be clashing against kern.maxfiles, but its value is 20000 while kern.openfiles is never above ~600.
Click to expand...
It is more likely that you run against ulimit. Default for max files in ulimit seems to be 1878741.

Otherwise, investigate whether the Python app maybe executes the closings with a delay, e.g. via a GC hook.
 
I don't know what caused the issue. The "solution" was creating a 2nd jail to process that huge amount of files, then from the host, a cron job restarts the jail every 10 minutes. Now the task was finished and the "processing" jail does not run anymore.
 
You must log in or register to reply here.
Back
Top