vmstat 5
in a terminal for a minute or two. This gives you a lot of useful numbers. See the vmstat(8) manual page for more options, and also refer to the tuning(7) manual page for a lot of valuable hints. lua-time-limit
from 5 secs to 0 to disable it (and also remove commands related to lua scripting as I will use redis only for LRU cache and the load average doesn't go up any more. lua-time-limit
when enabled it does a lot of time system calls that cause the higher load average.You can verify that by looking at the “Looks likelua-time-limit
when enabled it does a lot of time system calls that cause the higher load average.
sy
” column in the output of vmstat 5
. That's the average number of system calls per second. truss -cp PID
(replace “PID” with the actual ID number), then wait half a minute or so and press Ctrl-C. It will display a nice statistic about which system calls have been called how often, and how much time (cumulative) was spent inside those system calls. Note that you probably need to run the truss(1) command as root.# truss -cp 54378
^C
syscall seconds calls errors
write 0.000161744 6 0
sendmsg 0.000277396 6 0
recvmsg 0.000276278 12 6
read 0.000179902 12 6
kevent 8.751450549 23 0
gettimeofday 0.000187723 6 0
_umtx_op 8.747898601 11 0
------------- ------- -------
17.500432193 76 12
load averages: 0.54, 0.43, 0.37
; keep in mind:97.4% idle
I find running top -C -s 5
a great overall command option. vmstat -w 5
is another great command, and I have read using it with a 5 second internal is the best choice. That command tells you a lot of good stuff. Watch the 'b' column. That is the "blocked" column and it means a process was blocked from running due to lack of resources; if I remember right. For a mission critical machine I would argue that you should aways see a 0 there; meaning if you see a non-zero number occassionally, often, or consistantly you should investigate what is the bottleneck and consider upgrades. You "pi" and "po" columns are your paging stats. And the last column "id" is your cpu idle.Thevmstat -w 5
is another great command, and I have read using it with a 5 second internal is the best choice. That command tells you a lot of good stuff. Watch the 'b' column. That is the "blocked" column and it means a process was blocked from running due to lack of resources; if I remember right. For a mission critical machine I would argue that you should aways see a 0 there; meaning if you see a non-zero number occassionally, often, or consistantly you should investigate what is the bottleneck and consider upgrades. You "pi" and "po" columns are your paging stats. And the last column "id" is your cpu idle.
po
column is the more important one of the two, because it specifies the amount of page-out activity towards the swap partitions. If you get non-zero numbers here over longer periods of time, it might mean that you have a problem with the amount of RAM (or an application with a memory leak, or similar problem). On the other hand, the pi
column refers to all page-in activity, which includes loading code pages from executables and libraries. This is perfectly normal and does not indicate a problem.sr
column (scan rate). This is an indication of the “pressure” on the virtual memory system. The higher the number, the harder the VM system has to work to provide free pages of memory for applications that need them.It's the other way around. Page-outs are not a problem, lots of page-ins means you need more memory.Thepo
column is the more important one of the two, because it specifies the amount of page-out activity towards the swap partitions. If you get non-zero numbers here over longer periods of time, it might mean that you have a problem with the amount of RAM (or an application with a memory leak, or similar problem). On the other hand, thepi
column refers to all page-in activity, which includes loading code pages from executables and libraries. This is perfectly normal and does not indicate a problem.
As I explained, page-ins regularly happen from executables and libraries, when programs are started etc., and this is perfectly normal, no swap involved at all. Opposed to that, page-outs always go to the swap (the only exception is memory-mapped writable files, but not many programs do this). That doesn't have to mean a problem, especially if it happens only occasionally. But if it happens all the time in large quantities, it certainly indicates a memory shortage.It's the other way around. Page-outs are not a problem, lots of page-ins means you need more memory.
As I understood it, page-out or in always involves swap. Page-out is memory to swap, page-in is swap to memory. Given that definition, something can only be paged in if it was previously paged out.As I explained, page-ins regularly happen from executables and libraries, when programs are started etc., and this is perfectly normal, no swap involved at all.
That is not correct, I'm afraid. When a program is exec'ed, the executable and the libraries are mapped into the process image. As soon as pages of that image are used, they're paged in from their respective files (provided that they're not already in RAM, of course). This counts as page-ins for theAs I understood it, page-out or in always involves swap. Page-out is memory to swap, page-in is swap to memory. Given that definition, something can only be paged in if it was previously paged out.
pi
column of the vmstat
command. vmstat 5
in one window, and copy a bunch of files <= 8 MB (for example photos) with cp
in another window. Make sure they haven't been accessed recently, otherwise they might already be cached in RAM.procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr ad0 ad1 in sy cs us sy id
[...]
0 0 4 3.1G 550M 6 0 0 0 0 40 0 0 16 147 2161 0 1 99
0 0 4 3.1G 550M 29 0 0 0 17 41 1 1 16 180 2176 0 1 99
0 0 4 3.1G 303M 6371 0 784 0 227 40 802 763 1567 332 17468 0 10 90
0 0 4 3.1G 90M 7298 239 915 0 4102 4266 866 948 1817 188 19853 0 9 91
0 0 4 3.1G 99M 1103 0 87 0 2224 1830 110 62 219 1153 3841 1 2 97
0 0 4 3.1G 99M 38 0 0 0 1 50 0 0 37 637 2302 0 1 99
0 0 4 3.1G 99M 46 0 0 0 31 50 0 0 19 230 2197 0 1 99
You have to be careful there. In some applications, memory-mapping files for IO can be significantly more efficient than read() and write() calls. That's why some IO libraries use mmap() extensively. So even if a program doesn't look like it is doing it, it might do it indirectly through libraries. The examples I know of are some HPC (supercomputing) IO libraries, and some databases. Furthermore, mmap() can be used as a mechanism for allocating memory, and I don't know whether those page faults will be counted towards the pi/po totals. Now, I don't know how common those libraries are used in FreeBSD, in particular not in a desktop user scenario.... (the only exception is memory-mapped writable files, but not many programs do this).
Thanks for shattering my worldview. But I'm happy you did. It made me realize I need to schedule my reading material. I have the "Design and implementation ..." collecting dust on my bookshelf and haven't taken the time to actually read it.That is not correct, I'm afraid.