makewhatis: out of swap space

aragats

aragats

This happened on a small VPS running FreeBSD 11.1-RELEASE-p10 when performing weekly cron jobs.
The system has 512M of RAM and 1024M of swap, it's hard to believe it was not enough:
Code: 
% tail /var/log/messages
....
Jun  9 04:54:39 eu kernel: pid 15265 (makewhatis), uid 0, was killed: out of swap space
Jun  9 04:55:07 eu kernel: pid 609 (beam.smp), uid 543, was killed: out of swap space
The second process beam.smp belongs to net-im/ejabberd, which may have bugs/memory leaks etc, but why makewhatis was killed first?
There was no load at 4am, the system is running apache, and it's not a public website, just for personal use. I cannot imagine that makewhatis consumes so much memory. Of course, I can just disable it, never needed whatis...

Thanks for ideas!
 
When you consider what makewhatis actually does, it's not unreasonable to think that it could have caused this. I mean, your RAM is fairly small, and the utility must do a complete scan and tree build of the available manpages. I don't remember if makewhatis will process a discrete manpage base directory, and then moves on, or if it is trying to process the whole MANPATH environment in memory at once.

You could double or tripple your available swap space, but understand that it's gonna be slow slow slow processing to complete the task if it's using that much swap space.

One other point I just thought of...If you have corrupt or incorrect manpage files, then you might have a circular dependency in them that is gobbling up all the memory trying to resolve itself.
 
That all makes sense, thanks! Will experiment with swap space, but honestly speaking, I'll better disable that cron job, why I need it on that small VPS...
 
I tried running [I]makewhatis[/I] manually, it was killed by kernel after 10 minutes or so.
Also, last week I updated that system from 11.0 to 11.1, thus, it had to process new stuff.

Now, the question is how it's supposed to work in an embedded system where 512M of RAM is normal?
Why that job for [I]makewhatis[/I] is enabled by default?
 
aragats said:
I tried running [I]makewhatis[/I] manually, it was killed by kernel after 10 minutes or so.
Also, last week I updated that system from 11.0 to 11.1, thus, it had to process new stuff.

Now, the question is how it's supposed to work in an embedded system where 512M of RAM is normal?
Why that job for [I]makewhatis[/I] is enabled by default?
Click to expand...

My guess is that the RELEASE distro isn't specifically an embedded system release. In my world an embedded system is a kernel, required drivers, program daemon, and busybox. Thus the reason linux is so utterly unsuitable for real embedded work in recent years. If done by someone who knew what they were doing, you shouldn't see makewhatis or the manpages on a target embedded system. Yeah, you should strip out cron jobs that aren't needed. Do keep in mind though that if you really need logs then automated rotation of those logs is a necessity.
 
Well, maybe embedded term is not really correct, let's say ARM platform instead.
We have e.g. FreeBSD-11.1-RELEASE-arm-armv6-BEAGLEBONE.img.xz here, but BeagleBone Black has only 512M RAM.
I believe, that's the same RELEASE, isn't it? My point is that such things should not be enabled by default.
 
I have a FreeBSD 11.1-RELEASE-p10 amd64 VM instance running Apache and sendmail and a few milters with:

real memory = 536870912 (512 MB)
avail memory = 506380288 (482 MB)

$ swapinfo -h
Device 1K-blocks Used Avail Capacity
/dev/md99 524288 10M 502M 2%
Click to expand...

and it runs makewhatis with no issues at all.

I think you're really experiencing the "out of memory" bug which has been mentioned extensively in the arm mailing list.
 
No, no, I'm running it on amd64, not ARM. As I mentioned above, it could be caused by the upgrade 11.0->11.1 when makewhatis had to process new stuff.
 
Well my VM survived the upgrade from 10.4 to 11.1 without any oom issues and it has only 50% of the amount of swap you have. I wouldn't dismiss the arm issue as being unrelated. Not many people are running amd64 instances with under 1G of memory, apart from those with arm embedded systems.

Curiously, neither my rpi2 nor amd64 VM have encountered the issue (yet).
 
So run it again manually, and run top in a window to watch the virtual image size of the task grow.
 
You must log in or register to reply here.
Back
Top