iNotify for FreeBSD?

I

IT_Architect

Is there an iNotify for FreeBSD? Kqueue has to open every directory it watches, so too many directories will run it out of the max number of open files (typically 1024). That would never work on a web server.

Thanks!
 
I don't know, it seems there is none.
Anyway I believe this is the wrong forum, it is better placed in the (kernel) development area.
Moreover I cannot imagine why a single process should monitor more than 1024 directories, and since it has to monitor directories, probably splitting the process into hierarchical processes will work better.
 
fluca1978 said:
I cannot imagine why a single process should monitor more than 1024 directories
Click to expand...
Real-time anti-malware solutions all require it. It's the limitation they run into when attempting to implement kqueue. Not having the functionality of inotify is a show-stopper. I was hoping there was a way around this limitation in the newer FreeBSDs.

Thanks!
 
will run it out of the max number of open files (typically 1024)
Click to expand...

Where do you get the 1024 limit?

Code: 
[~]% sysctl -d kern.maxfiles
kern.maxfiles: Maximum number of files
[~]% sysctl -d kern.maxfilesperproc
kern.maxfilesperproc: Maximum files allowed open per process

[~]% sysctl kern.maxfiles
kern.maxfiles: 12328
[~]% sysctl kern.maxfilesperproc
kern.maxfilesperproc: 11095

You can increase these further, but these are the defaults on my FreeBSD 8.2 and 9.0 systems...
 
Real-time anti-malware for FreeBSD? I take it's for checking Windows malware on a fileserver running FreeBSD :P
 
Carpetsmoker said:
Where do you get the 1024 limit?...You can increase these further, but these are the defaults on my FreeBSD 8.2 and 9.0 systems...
Click to expand...

I was reading here. http://git-annex.branchable.com/design/assistant/inotify/
and here: http://en.wikipedia.org/wiki/Gamin
"Under Linux, Gamin uses inotify or dnotify to monitor filesystem activity. Under FreeBSD, Gamin uses kqueue/kevent as the native FreeBSD kernel event notification mechanism, but in this case kqueue has one major drawback as compared to inotify: inotify is the filename-based monitoring facility and kqueue uses file descriptors for identification of monitored files, so, under FreeBSD, gam_server has to open each file in the monitored directory. This can be an issue when monitoring directories with many files stored, because system could easily reach its kernel limits on maximum file descriptor count (kern.maxfiles and kern.maxfilesperproc)."
Click to expand...

SirDice said:
No, iNotify is a Linux kernel feature.
Click to expand...

wblock@ said:
The mailing lists (maybe freebsd-current) are probably a better source for this type of information.
Click to expand...

Perhaps so. I never heard if inotify for FreeBSD, but while searching I found this http://www.freebsd.org/cgi/man.cgi?query=inotify&apropos=0&sektion=0&manpath=SuSE+Linux/i386+11.3&format=html a post on this forum. http://forums.freebsd.org/showthread.php?t=29988
and this: https://github.com/skirge/fsnotifier-freebsd/blob/master/inotify.c which is based on kqueue. I was hoping.

kpa said:
Real-time anti-malware for FreeBSD? I take it's for checking Windows malware on a fileserver running FreeBSD :P
Click to expand...

No, the problem is bots guessing passwords and placing exploits on their pages. You can have users come up with passwords that meet all of the complexity requirements but the bot finds them because they use things that are easy for them to remember. When their site gets blacklisted, and it's my fault. Firewall policies are problematic because bots know how to stay under the radar. If you set the policy below the robot, it also blocks customers.

There is a program designed to catch web exploits. Linux Malware Detector (LMD). I deployed that. The problem is, I put a known exploit out there that I saved from a previous incident, and it didn't find it, while ClamAV, which is not designed for that type of exploit, did find it. So I'm just trying to come up with ways to catch things before they get blacklisted by Google and Yahoo!/Bing. ClamAV works fine for a daily scan, and even hourly with parameters that check only the files that have changed in the past hour. However, the right way is to scan files the moment they are created or change. IMHO, this has become a necessity in this day and age.

Thanks all!
 
iNotify would also be useful for all the cloud synchronization services that start to become popular. They all have the same problem that they need to watch folders that get automatically synchronized to the cloud when they change or files are added.
 
IT_Architect said:
There is a program designed to catch web exploits. Linux Malware Detector (LMD). I deployed that. The problem is, I put a known exploit out there that I saved from a previous incident, and it didn't find it, while ClamAV, which is not designed for that type of exploit, did find it.
Click to expand...
Unfortunately all those types of programs all work signature based. This means it will never catch everything. If I write a brand new exploit (or something specifically targeted at your site) these programs will never catch it. Just keep a sharp eye on your logs, analyse them daily. Filter out the obvious crap and see what's left. If you do that on a regular basis you will quickly see things that are out of place, investigate those.

So I'm just trying to come up with ways to catch things before they get blacklisted by Google and Yahoo!/Bing.
Click to expand...
Make sure you set up an abuse email address or some other way people can use to complain. Read them, investigate and act on them as quickly as possible. You usually only get put on banlists when there's no way to comlain or you simply don't act on any abuse complaints.
 
From a user perspective having to enable audit seems wrong. The audit implementation could probably be a good example on how to get the information in the kernel but it needs a proper API for the userland.
 
decke said:
From a user perspective having to enable audit seems wrong. The audit implementation could probably be a good example on how to get the information in the kernel but it needs a proper API for the userland.
Click to expand...
I suppose it would also pose a problem with jailed processes and other non-privileged processes, but I made the suggestion given @IT_Architect's goal of server security. I certainly wouldn't use audit as a substitute for inotify myself because every occasion I've had to monitor files or directories in real time only involved a handful of each.

Kevin Barry
 
Last edited by a moderator:
fswatch, albeit sporting for now a GNU license, could perhaps help? https://github.com/emcrisostomo/fswatch

But, regarding FreeBSD, the README says:

The kqueue monitor, available on any *BSD system featuring kqueue, requires a file descriptor to be opened for every file being watched. As a result, this monitor scales badly with the number of files being observed and may begin to misbehave as soon as the fswatch process runs out of file descriptors. In this case, fswatch dumps one error on standard error for every file that cannot be opened.

(PS: I also posted this on similar Thread 42792)
 
  • Thanks
Reactions: taz
I personally would like to see an iNotify API. I know its a Linux thing but it is very convenient.

In many cases I just use Go for these types of problems, it provides an fsnotify interface across platforms that is "good enough".
 
b7j0c said:
I personally would like to see an iNotify API. I know its a Linux thing but it is very convenient.

In many cases I just use Go for these types of problems, it provides an fsnotify interface across platforms that is "good enough".
Click to expand...

There's one in the ports tree. From the pkg-descr:

This library provides inotify-compatible interface for applications,
that need to monitor changes happening in a filesystem. It can be useful
when porting Linux applications, which often use inotify interface.

The IN_OPEN, IN_CLOSE_WRITE and IN_CLOSE_NOWRITE events are not yet
implemented, so the relevant tests are known to fail.

Author: Dmitry Matveev <me@dmitrymatveev.co.uk>
WWW: https://github.com/dmatveev/libinotify-kqueue
​
If you install from pkg it even gives you a nice little warning that you may want to bump the kern.maxfiles value
​
 
Could Dtrace help here? I'm sure there would be a way to get it to alert on file access, though I don't know how you would integrate that with other applications.
 
Much like the OP I have been searching for program that will monitor a file for changes.

After much searching I found a simple and excellent solution in this page: http://doc.geoffgarside.co.uk/kqueue/
Scroll to the bottom of the page and select: EVFILT_VNODE
There you will find the c source code for the excellent utility. Compile and enjoy!

By the way, I did try fswatch: https://github.com/emcrisostomo/fswatch. But could only get it to compile in FreeBSD 10 and I needed something that worked in v9. The issue with fswatch is that is written in c++ and the c++ compiler in FreeBSD 9.x is not working properly.

Just for clarification, FreeBSD has an fswatch in the ports but it is not related to the fswatch I just mentioned. The fswatch in the ports is a hash generator for files...security/fswatch
 
decke said:
iNotify would also be useful for all the cloud synchronization services that start to become popular. They all have the same problem that they need to watch folders that get automatically synchronized to the cloud when they change or files are added.
Click to expand...

Agreed. Some sort of 'Notify' is getting more and more desirable/needed. Cloud sync is taking off like crazy.
 
But those all are all just frontends for kqueue, so don't solve the problems of having to open every directory it's watching.
 
You must log in or register to reply here.
Back
Top