• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

Swap warnings after upgrade to 11.2

mickey

Well-Known Member

Thanks: 57
Messages: 371

#1
I am getting these warnings about swap space since I upgraded an old i386 machine from FreeBSD 11.1-p10 to 11.2 from source tree:
Code:
warning: total configured swap (524288 pages) exceeds maximum recommended amount (509152 pages).
warning: increase kern.maxswzone or reduce amount of swap.
warning: total configured swap (1048576 pages) exceeds maximum recommended amount (509152 pages).
warning: increase kern.maxswzone or reduce amount of swap.
The machine has 1G RAM installed and has two GPT partitioned hard drives, each with a 2G swap area. A ratio of 1:4 is probably more than sufficient but I would not call it excessive. And as you can see even one 2G partition already exceeds the recommended amount. The setting of kern.maxswzone also does not appear too small and I don't think I've ever messed with the default value:
Code:
kern.maxswzone: 36175872
Any idea why I now get those warnings? Did I maybe miss to upgrade something in my cutom kernel config or elsewhere?
 

lebarondemerde

Daemon

Thanks: 591
Messages: 1,365

#2
I am also getting this message on my home server (2GB of RAM, 4B of SWAP) with 11.2; however I did not invested time to investigate it yet. This is my default in there:

Code:
kern.maxswzone: 0
(EDIT)

So, I set mine to kern.maxswzone="15728640" and that shut the message. Why, I don't know.
 

chrbr

Aspiring Daemon

Thanks: 225
Messages: 627

#3
I have got that warnings as well, may be it has been after upgrade from FreeBSD10.3 to FreeBSD11.0. At that time I have had 2G of RAM and 8G of swap space. I have searched in the internet and found that too much swap space can be contra productive. I have reduced the size of the swap to 6G and never run out of swap space working with the system.
 

trev

Aspiring Daemon

Thanks: 103
Messages: 596

#4
From the arm mailing list via Mark Millard:

"man 8 loader" and its kern.maxswzone material,

Note that swap metadata can be fragmented, which means that
the system can run out of space before it reaches the
theoretical limit. Therefore, care should be taken to not
configure more swap than approximately half of the
theoretical maximum.
 

mickey

Well-Known Member

Thanks: 57
Messages: 371

#5
The more interesting part is this:
Code:
                   Limits the amount of KVM to be used to hold swap metadata,
                   which directly governs the maximum amount of swap the
                   system can support, at the rate of approximately 200 MB of
                   swap space per 1 MB of metadata.  This value is specified
                   in bytes of KVA space.  If no value is provided, the system
                   allocates enough memory to handle an amount of swap that
                   corresponds to eight times the amount of physical memory
                   present in the system.
Considering above, 2G of swap or even 4G is hardly anywhere near eight times the amount of physical memory present in the system. Also when my kern.maxswzone reads 36175872 it should roughly support 36175872 / 1048576 * 200 = 6900 Mb of swap space, according to above mentioned math. But none of this explains why a system with 1G ram complains with 2G swap, which is just ridiculous.
 

trev

Aspiring Daemon

Thanks: 103
Messages: 596

#6
Code:
sysctl kern.maxswzone
kern.maxswzone: 0
The default _is_ zero, so:

If no value is provided, the system allocates enough memory to handle an amount of swap that
corresponds to eight times the amount of physical memory present in the system.
The above statement now appears incorrect (as of FreeBSD 9.2 - change to 0 was merged into 9-stable in September 2012).
 

mickey

Well-Known Member

Thanks: 57
Messages: 371

#7
Code:
sysctl kern.maxswzone
kern.maxswzone: 0
The default _is_ zero, so:
I have absolutely no idea, where the system gets the value of 36175872 from, which is the value sysctl reports for kern.maxswzone when the system is running. Maybe i386 has a different default value than amd64? I double checked /boot/loader.conf as well as /etc/sysctl.conf it's not in there. I even grepped through all the files in /boot and /etc and found nothing. My custom kernel config also contains nothing swap related, the only VM related stuff in there is some ZFS tuning, namely:
Code:
options         KVA_PAGES=512
options         KSTACK_PAGES=4
But that aside, the value should still be large enough to allow for roughly 6900 Mb swap space. A single 2G swap partition should not even exceed half of that maximum, yet the system complains once for the first 2G swap partition and once more for the second.
 

PMc

Member

Thanks: 24
Messages: 84

#8
Same here after upgrading 11.1-p10 to 11.2. i386, 2G ram, 8G swap.
warning: total configured swap (2097152 pages) exceeds maximum recommended amount (1022784 pages).
warning: increase kern.maxswzone or reduce amount of swap.


Increasing kern.maxswzone does not change anything. Default is:
kern.maxswzone: 36175872
(on amd64 the default appears to be 0)

This is hardcoded in sys/i386/include/param.h, independent of installed ram:
Code:
 * 276 is sizeof(struct swblock), but we do not always have a definition
* in scope for struct swblock, so we have to hardcode it.  Each struct
* swblock holds metadata for 32 pages, so in theory, this is enough for
* 16 GB of swap.  In practice, however, the usable amount is considerably
* lower due to fragmentation.
*/
#ifndef VM_SWZONE_SIZE_MAX
#define VM_SWZONE_SIZE_MAX      (276 * 128 * 1024)
#endif
Various posts (and the code, and my experience) appear to say that kern.maxswzone can only be set to reduce the value, not increase it. So it seems at least the warning message is bogus.
Anyway, supporting a maximum swap of 2*mem seems too small. So something else is also wrong here.

Glancing through the code shows more severe changes - and the math triggering the warning seems to be changed so that it now triggers at a 1/4 of the former value. I might tend to assume that the error is on the side of the warning, not the behaviour.

Anyway, it seems possible to watch the actual usage:
Code:
# vmstat -z | grep swblk
ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP
swblk:                  136, 492826,   36464,    6978,  337344,   0,   0
 

mickey

Well-Known Member

Thanks: 57
Messages: 371

#9
Same here after upgrading 11.1-p10 to 11.2. i386, 2G ram, 8G swap.
warning: total configured swap (2097152 pages) exceeds maximum recommended amount (1022784 pages).
warning: increase kern.maxswzone or reduce amount of swap.


Increasing kern.maxswzone does not change anything. Default is:
kern.maxswzone: 36175872
(on amd64 the default appears to be 0)
In the meantime I also upgraded my desktop machine to RELENG-11.2 and can confirm that kern.maxswzone is indeed 0 on an amd64 system.

This is hardcoded in sys/i386/include/param.h, independent of installed ram:
Code:
 * 276 is sizeof(struct swblock), but we do not always have a definition
* in scope for struct swblock, so we have to hardcode it.  Each struct
* swblock holds metadata for 32 pages, so in theory, this is enough for
* 16 GB of swap.  In practice, however, the usable amount is considerably
* lower due to fragmentation.
*/
#ifndef VM_SWZONE_SIZE_MAX
#define VM_SWZONE_SIZE_MAX      (276 * 128 * 1024)
#endif
Various posts (and the code, and my experience) appear to say that kern.maxswzone can only be set to reduce the value, not increase it. So it seems at least the warning message is bogus.
Anyway, supporting a maximum swap of 2*mem seems too small. So something else is also wrong here.

Glancing through the code shows more severe changes - and the math triggering the warning seems to be changed so that it now triggers at a 1/4 of the former value. I might tend to assume that the error is on the side of the warning, not the behaviour.
I remember having read something similar somewhere, that the value can only be decreased but not increased, but haven't tested it myself, mainly because I don't even think the value is too small in the first place. I agree that this must be the (probably erroneous) result of code changes to /sys/vm/swap_pager.c and friends.

Anyway, it seems possible to watch the actual usage:
Code:
# vmstat -z | grep swblk
ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP
swblk:                  136, 492826,   36464,    6978,  337344,   0,   0
Do I need to be concerned about the low number of free blocks?
Code:
ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP
swblk:                   72, 127288,    2489,      31,    2752,   0,   0
 

PMc

Member

Thanks: 24
Messages: 84

#11
Do I need to be concerned about the low number of free blocks?
Code:
ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP
swblk:                   72, 127288,    2489,      31,    2752,   0,   0
One would need to read about the working of the uma thing (but I have to catch a flight just now) - I would suppose (with no certainty) that the FREE column is space that was already used and stays allocated until mem shortage. I would suppose that reaching the LIMIT is what would give a problem.

But yesterday my desktop (amd64, 4g ram, 10g swap, no warning) has slowly run into complete unresponsiveness after opening some 50 tabs in Firefox. Such has never happened before, but Firefox ESR was upgraded to 60 which now consumes immense mem - and just now I see the these (lots of) errors in the log:
Jul 10 16:34:26 <kern.crit> disp kernel: swap_pager: I/O error - pageout failed; blkno 796755,size 32768, error 12
If that were an actual disk I/O error, and reset was pushed, how could these errors have made it to /var/log/messages ? And error12 is ENOMEM. So there seems to be more crap upcoming...
 

Datapanic

Active Member

Thanks: 123
Messages: 228

#12
Related, I filed a bug report for FreeBSD 11.2 amd64 VM's failing to boot because if the VM was configured with 256MB of ram, /etc/rc.conf would not be written to disk during the install.

I also noted that the swap size would be different if I configured a VM with 384MB of ram and either a 12GB or 32GB disk. if I set a 12GB disk, then swap was set at 628736K but if I set a 32GB disk, then swap was set at 1677312K.

To reproduce...

ESXi 5.5
VM with 1 CPU
12GB DISK
256MB RAM
GPT default partitions
DOC, SCR and PORTS

reboot after install and there is no /etc/rc.conf.

Works when setting ram to 384MB

:fishy:
 

Lamia

Active Member

Thanks: 20
Messages: 143

#13
Welcome to the world of FreeBSD 11.2. I was welcomed by the "get_swap_......." messages too after an upgrade. It seems to be a nicer substitute for "swap_pager_getswapspace". Then, I was completely locked out via ssh. All fixes for the SSH failed; so I decided [and needed] to build kernel/world.

I have successfully built 11.1 world/kernel on a local machine running the same version without any hacks, but not 11.2 on a VM [already upgraded to 11.2]. The most problem is with one or two header files - xmmintrin.h for 11.1 & wmmintrin.h for 11.2 on the VM. Most suggestions from the FreeBSD bugzilla here failed. Basically, copying the clang files/folders to /usr/include did not work. I am now trying to use the lang/clang-devel [which comes with llvm-devel as a dependency]. Of course, that looks more promising as a devel/header package.
 

mickey

Well-Known Member

Thanks: 57
Messages: 371

#14
One would need to read about the working of the uma thing (but I have to catch a flight just now) - I would suppose (with no certainty) that the FREE column is space that was already used and stays allocated until mem shortage. I would suppose that reaching the LIMIT is what would give a problem.
Yes, it appears to be that way indeed. After a fresh boot I get 0 free blocks:
Code:
ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP
swblk:                   72, 256200,       0,       0,       0,   0,   0
I just upgraded the machine to 2G ram for the sake of testing, but it's still complaining about the size of the swap partitions. Well at least now I only get one such warning. Man this thing is picky:
Code:
warning: total configured swap (1048576 pages) exceeds maximum recommended amount (1024800 pages).
warning: increase kern.maxswzone or reduce amount of swap.
Interestingly now the swap size reported by swapinfo and top seems to match. I could have sworn that with only 1G ram top displayed a slightly smaller amount than swapinfo:
Code:
swapinfo -h:

Device          1K-blocks     Used    Avail Capacity
/dev/ada0p2       2097152       0B     2.0G     0%
/dev/ada1p2       2097152       0B     2.0G     0%
Total             4194304       0B     4.0G     0%

top:

Swap: 4096M Total, 4096M Free
But yesterday my desktop (amd64, 4g ram, 10g swap, no warning) has slowly run into complete unresponsiveness after opening some 50 tabs in Firefox. Such has never happened before, but Firefox ESR was upgraded to 60 which now consumes immense mem - and just now I see the these (lots of) errors in the log:
Jul 10 16:34:26 <kern.crit> disp kernel: swap_pager: I/O error - pageout failed; blkno 796755,size 32768, error 12
If that were an actual disk I/O error, and reset was pushed, how could these errors have made it to /var/log/messages ? And error12 is ENOMEM. So there seems to be more crap upcoming...
I hope not, but I guess only time will tell. How did you manage to open 50(!) tabs in firefox anyways? I guess my personal garbage collector would have kicked in long before doing so :-D
 

PMc

Member

Thanks: 24
Messages: 84

#15
I hope not, but I guess only time will tell. How did you manage to open 50(!) tabs in firefox anyways? I guess my personal garbage collector would have kicked in long before doing so :-D
I dont know how many there are actually - it's just how I am doing things. Say, I search for some matter, find a bunch of interesting links, and just open them to new tabs for later evaluation. Then I might get a mail and be pushed to some other matter, grab a free screen (icewm) and dive into that. So, at any time there may be a couple of browser windows on different screens with various tabs where I might continue reading later, and while these are not actively used, they are supposed to move into paging space.
 

Lamia

Active Member

Thanks: 20
Messages: 143

#16
It is so easy to have firefox with over 50tabs running. Using multiple profiles of firefox (by running "firefox --no-remote -P") is one example of doing so. A user could install several categories of add-ons and themes on the various profiles according to their needs. With that, they can open tonnes of websites on each one of them.

A typical situation is when one is trying to troubleshoot a failing service, package, etc and needs to catch up with the other distractions of life - Sports (NBA/Tour de France/AFL/Wimbledon/FiFa@Russia/etc).
 

PMc

Member

Thanks: 24
Messages: 84

#17
Didn't find a solution to the swap issue yet, but found that the memory greediness of Firefox can be tuned.
There's a switch in preferences, how many content worker processes it should run, and these are not threads, but full-sized processes, and they get *BIG*. Since the workload seems to be distributed among them, there is no point in having more than real mem can fit. As soon as the amount of tabs leads to paging, one is better off with only one worker process.
 
Top