• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

iNotify for FreeBSD?

IT_Architect

Member

Thanks: 4
Messages: 86

#1
Is there an iNotify for FreeBSD? Kqueue has to open every directory it watches, so too many directories will run it out of the max number of open files (typically 1024). That would never work on a web server.

Thanks!
 

fluca1978

Aspiring Daemon

Thanks: 70
Messages: 735

#2
I don't know, it seems there is none.
Anyway I believe this is the wrong forum, it is better placed in the (kernel) development area.
Moreover I cannot imagine why a single process should monitor more than 1024 directories, and since it has to monitor directories, probably splitting the process into hierarchical processes will work better.
 

IT_Architect

Member

Thanks: 4
Messages: 86

#3
fluca1978 said:
I cannot imagine why a single process should monitor more than 1024 directories
Real-time anti-malware solutions all require it. It's the limitation they run into when attempting to implement kqueue. Not having the functionality of inotify is a show-stopper. I was hoping there was a way around this limitation in the newer FreeBSDs.

Thanks!
 

Carpetsmoker

Daemon

Thanks: 180
Messages: 1,010

#4
will run it out of the max number of open files (typically 1024)
Where do you get the 1024 limit?

Code:
[~]% sysctl -d kern.maxfiles
kern.maxfiles: Maximum number of files
[~]% sysctl -d kern.maxfilesperproc
kern.maxfilesperproc: Maximum files allowed open per process

[~]% sysctl kern.maxfiles
kern.maxfiles: 12328
[~]% sysctl kern.maxfilesperproc
kern.maxfilesperproc: 11095
You can increase these further, but these are the defaults on my FreeBSD 8.2 and 9.0 systems...
 

kpa

Beastie's Twin

Thanks: 1,682
Messages: 6,084

#7
Real-time anti-malware for FreeBSD? I take it's for checking Windows malware on a fileserver running FreeBSD :p
 

IT_Architect

Member

Thanks: 4
Messages: 86

#8
Carpetsmoker said:
Where do you get the 1024 limit?...You can increase these further, but these are the defaults on my FreeBSD 8.2 and 9.0 systems...
I was reading here. http://git-annex.branchable.com/design/assistant/inotify/
and here: http://en.wikipedia.org/wiki/Gamin
"Under Linux, Gamin uses inotify or dnotify to monitor filesystem activity. Under FreeBSD, Gamin uses kqueue/kevent as the native FreeBSD kernel event notification mechanism, but in this case kqueue has one major drawback as compared to inotify: inotify is the filename-based monitoring facility and kqueue uses file descriptors for identification of monitored files, so, under FreeBSD, gam_server has to open each file in the monitored directory. This can be an issue when monitoring directories with many files stored, because system could easily reach its kernel limits on maximum file descriptor count (kern.maxfiles and kern.maxfilesperproc)."
SirDice said:
No, iNotify is a Linux kernel feature.
wblock@ said:
The mailing lists (maybe freebsd-current) are probably a better source for this type of information.
Perhaps so. I never heard if inotify for FreeBSD, but while searching I found this http://www.freebsd.org/cgi/man.cgi?query=inotify&apropos=0&sektion=0&manpath=SuSE+Linux/i386+11.3&format=html a post on this forum. http://forums.freebsd.org/showthread.php?t=29988
and this: https://github.com/skirge/fsnotifier-freebsd/blob/master/inotify.c which is based on kqueue. I was hoping.

kpa said:
Real-time anti-malware for FreeBSD? I take it's for checking Windows malware on a fileserver running FreeBSD :p
No, the problem is bots guessing passwords and placing exploits on their pages. You can have users come up with passwords that meet all of the complexity requirements but the bot finds them because they use things that are easy for them to remember. When their site gets blacklisted, and it's my fault. Firewall policies are problematic because bots know how to stay under the radar. If you set the policy below the robot, it also blocks customers.

There is a program designed to catch web exploits. Linux Malware Detector (LMD). I deployed that. The problem is, I put a known exploit out there that I saved from a previous incident, and it didn't find it, while ClamAV, which is not designed for that type of exploit, did find it. So I'm just trying to come up with ways to catch things before they get blacklisted by Google and Yahoo!/Bing. ClamAV works fine for a daily scan, and even hourly with parameters that check only the files that have changed in the past hour. However, the right way is to scan files the moment they are created or change. IMHO, this has become a necessity in this day and age.

Thanks all!
 

decke

Member
Developer

Thanks: 4
Messages: 20

#9
iNotify would also be useful for all the cloud synchronization services that start to become popular. They all have the same problem that they need to watch folders that get automatically synchronized to the cloud when they change or files are added.
 

SirDice

Administrator
Staff member
Administrator
Moderator

Thanks: 6,127
Messages: 27,000

#11
IT_Architect said:
There is a program designed to catch web exploits. Linux Malware Detector (LMD). I deployed that. The problem is, I put a known exploit out there that I saved from a previous incident, and it didn't find it, while ClamAV, which is not designed for that type of exploit, did find it.
Unfortunately all those types of programs all work signature based. This means it will never catch everything. If I write a brand new exploit (or something specifically targeted at your site) these programs will never catch it. Just keep a sharp eye on your logs, analyse them daily. Filter out the obvious crap and see what's left. If you do that on a regular basis you will quickly see things that are out of place, investigate those.

So I'm just trying to come up with ways to catch things before they get blacklisted by Google and Yahoo!/Bing.
Make sure you set up an abuse email address or some other way people can use to complain. Read them, investigate and act on them as quickly as possible. You usually only get put on banlists when there's no way to comlain or you simply don't act on any abuse complaints.
 

decke

Member
Developer

Thanks: 4
Messages: 20

#12
From a user perspective having to enable audit seems wrong. The audit implementation could probably be a good example on how to get the information in the kernel but it needs a proper API for the userland.
 

ta0kira

Active Member

Thanks: 22
Messages: 157

#13
decke said:
From a user perspective having to enable audit seems wrong. The audit implementation could probably be a good example on how to get the information in the kernel but it needs a proper API for the userland.
I suppose it would also pose a problem with jailed processes and other non-privileged processes, but I made the suggestion given @IT_Architect's goal of server security. I certainly wouldn't use audit as a substitute for inotify myself because every occasion I've had to monitor files or directories in real time only involved a handful of each.

Kevin Barry
 
Last edited by a moderator:

priyadarshan

Active Member

Thanks: 33
Messages: 115

#14
fswatch, albeit sporting for now a GNU license, could perhaps help? https://github.com/emcrisostomo/fswatch

But, regarding FreeBSD, the README says:

The kqueue monitor, available on any *BSD system featuring kqueue, requires a file descriptor to be opened for every file being watched. As a result, this monitor scales badly with the number of files being observed and may begin to misbehave as soon as the fswatch process runs out of file descriptors. In this case, fswatch dumps one error on standard error for every file that cannot be opened.

(PS: I also posted this on similar Thread 42792)
 

b7j0c

Member

Thanks: 4
Messages: 47

#15
I personally would like to see an iNotify API. I know its a Linux thing but it is very convenient.

In many cases I just use Go for these types of problems, it provides an fsnotify interface across platforms that is "good enough".
 

sweeney

New Member

Thanks: 2
Messages: 2

#16
I personally would like to see an iNotify API. I know its a Linux thing but it is very convenient.

In many cases I just use Go for these types of problems, it provides an fsnotify interface across platforms that is "good enough".
There's one in the ports tree. From the pkg-descr:

This library provides inotify-compatible interface for applications,
that need to monitor changes happening in a filesystem. It can be useful
when porting Linux applications, which often use inotify interface.

The IN_OPEN, IN_CLOSE_WRITE and IN_CLOSE_NOWRITE events are not yet
implemented, so the relevant tests are known to fail.

Author: Dmitry Matveev <me@dmitrymatveev.co.uk>
WWW: https://github.com/dmatveev/libinotify-kqueue
If you install from pkg it even gives you a nice little warning that you may want to bump the kern.maxfiles value
 

tanked

Active Member

Thanks: 1
Messages: 111

#17
Could Dtrace help here? I'm sure there would be a way to get it to alert on file access, though I don't know how you would integrate that with other applications.
 

mrtonyg

Member

Thanks: 2
Messages: 29

#19
Much like the OP I have been searching for program that will monitor a file for changes.

After much searching I found a simple and excellent solution in this page: http://doc.geoffgarside.co.uk/kqueue/
Scroll to the bottom of the page and select: EVFILT_VNODE
There you will find the c source code for the excellent utility. Compile and enjoy!

By the way, I did try fswatch: https://github.com/emcrisostomo/fswatch. But could only get it to compile in FreeBSD 10 and I needed something that worked in v9. The issue with fswatch is that is written in c++ and the c++ compiler in FreeBSD 9.x is not working properly.

Just for clarification, FreeBSD has an fswatch in the ports but it is not related to the fswatch I just mentioned. The fswatch in the ports is a hash generator for files...security/fswatch
 

PacketMan

Aspiring Daemon

Thanks: 132
Messages: 855

#20
iNotify would also be useful for all the cloud synchronization services that start to become popular. They all have the same problem that they need to watch folders that get automatically synchronized to the cloud when they change or files are added.
Agreed. Some sort of 'Notify' is getting more and more desirable/needed. Cloud sync is taking off like crazy.
 

Carpetsmoker

Daemon

Thanks: 180
Messages: 1,010

#23
But those all are all just frontends for kqueue, so don't solve the problems of having to open every directory it's watching.
 

Andrew Schmidt

New Member

Thanks: 2
Messages: 7

#25
Top