swap_pager_getswapspace: failed

W

wattie

I get lots of these in /var/log/messages:

Code: 
Jun 13 00:33:38 srv2 kernel: swap_pager_getswapspace(5): failed
Jun 13 00:33:38 srv2 kernel: swap_pager_getswapspace(8): failed
Jun 13 00:33:38 srv2 kernel: swap_pager_getswapspace(5): failed

Here is the output of the top command (sorted by size):

Code: 
last pid: 71242;  load averages:  4.38,  3.96,  3.57                                                                                                                                                                up 112+03:14:36 00:33:41
244 processes: 4 running, 240 sleeping
CPU:  8.2% user,  5.9% nice, 12.8% system,  0.0% interrupt, 73.0% idle
Mem: 3038M Active, 24G Inact, 3170M Wired, 1598M Buf, 484M Free
Swap: 4096M Total, 3843M Used, 253M Free, 93% Inuse

  PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND
73908 mysql        65  20    0  8948M  2571M select  8  26.6H  82.25% mysqld
96273 root          2  20    0   734M   535M select  0 125.1H   0.00% clamd
 7743 root          1  20    0   259M   168M select  5   3:06   0.00% perl
57615 apache      147  52    0   226M   102M piperd  3   0:18   1.41% httpd
...

Despite that error in the logs, I don't see anything going wrong - all services appear to be running fine.

As you can see I have lots of Inact memory. How do I find out what is using the swap and why?
 
wattie said:
Despite that error in the logs, I don't see anything going wrong
Click to expand...
You're running out of swap space. It's 93% in use.

wattie said:
How do I find out what is using the swap and why?
Click to expand...
The machine looks to have 8GB and MySQL is using 8 or more. That's never a good idea. There's just nothing left for everything else. Tune MySQL to use less memory, never allow it to use more memory than physically is in the box. Typically you configure about 2/3 memory for MySQL. If you have lots of other things running on it you may want to lower that number.
 
Please take a better look at the top. I have 32GB of ram.

Code: 
root@srv2:~ # sysctl hw.physmem
hw.physmem: 34207567872

Code: 
root@srv2:~ # grep memory /var/run/dmesg.boot
real memory  = 34359738368 (32768 MB)
avail memory = 33138290688 (31603 MB)
 
Oops, didn't have my coffee yet. You appear to have a lot of inactive memory. The reason it's using swap is because most of your memory is inactive, which cannot be freed for other purposes. I'd look into what's eating the inactive memory, swap usage is a consquence not the cause. Some application could have a memory leak.
 
Oook... and I have no clue where to start from, hehe :)

I will give "sysctl vm.defer_swapspace_pageouts=1" a try and will see if it will change anything...
 
wattie said:
Oook... and I have no clue where to start from, hehe :)

I will give "sysctl vm.defer_swapspace_pageouts=1" a try and will see if it will change anything...
Click to expand...

I had the same problem on my FreeBSD 11-RELEASE (8 G of RAM) when I am building ports with Synth and I did try your settings and it works :).
 
Yep, after I enabled that, I have a constant

Code: 
Swap: 4096M Total, 96M Used, 4000M Free, 2% Inuse
 
I tried your fix and I get the following:
[root@sting /home/filippo]# sysctl vm.defer_swapspace_pageouts=1
sysctl: unknown oid 'vm.defer_swapspace_pageouts'
Filippo
 
I have similar issues. I am on 12-RELEASE and use packages from the 'latest' repository.

I have KDE5 installed on an old hdd and after pkg update -f and pkg upgrade KDE didn't finish starting and ended with a black screen and a cursor. First I thought it had something todo with the hdd because it had bad sectors.

Yesterday I did a clean installation of 12-RELEASE-p3 and KDE5 on an other drive. Same issues, even though the installer had created (a bigger) swapspace because the size of the drive is bigger.

I am using the nvidia-driver and pkg only.
 
I have never had such issue speaking of desktop installation

I usually setup a swap size in the range of x1.5 to x2 the ram installed as recommended by the handbook

If one has 32 GB of ram, one should theoretically setup a swap of 48 GB at a minimum.....
For 8 GB, a common size of ram today, swap should be at 12 GB minimum

In some cases, swap size may have to be increased to more than x2, but this is in a server context, running for example big databases.

I can encounter such issue with a "portable freesbsd usb stick", but in such case I deactivate the swap

Using Swap with USB stick generally slows down the system because it generates too much IO traffic
Sometimes the swap is full because USB key encounters too much IO traffic, and has some problems to refresh the swap partition.
And as USB Stick is just a recovery key for general purpose in order to repair a broken system, I can't adapt the swap size... so the solution is to deactivate the swap in such context.

So yes... this kind of message may in some cases reveal some read/write issues with hard drive, especially some very slow hard drives
 
I tried to disable the swap partition and that resulted in that both plasmashell and kwin_x11 where killed (out of swapspace).

I kept the swapspace disabled and created a swapfile of twice the size of the installed memory and mounted it as md99. Same result, plasmashell was killed (out of swapspace).

I'm out of ideas right now, maybe this is Plasma/KDE5 related!?
 
rsronin said:
I tried to disable the swap partition and that resulted in that both plasmashell and kwin_x11 where killed (out of swapspace).

I kept the swapspace disabled and created a swapfile of twice the size of the installed memory and mounted it as md99. Same result, plasmashell was killed (out of swapspace).

I'm out of ideas right now, maybe this is Plasma/KDE5 related!?
Click to expand...


You use a swap file.
I personally don't use swap file, I use a dedicated swap partition.
This might be the issue...

When you create a swapfile, it also depends on how you create this swap file.
It is known that swap needs contiguous space.... so if you set a swap file with a maximum but variable size, if your system runs out of space, it may have some issues to find contiguous space and swapping process goes to error.
If you have defined a fixed swap size, problem shouldn't occur because system should have written a file filled with zero to reserve the contiguous space

According to me using a swap file is not the good solution
You must set a swap partition that is the best guaranty that contiguous max swap size will be always here
You can possibly add a swap file as a complement to a swap partition


PS : Note that UFS, like many other Linuxes filesystem, theoretically is not fragmenting files as NTFS does. But this is theoretical and factually wrong. There is always a little fragmentation, and in any case when available space is reduced, technically UFS has no other solution than fragmenting files. When we say that a given filesystem is not fragmenting this only means that the filesystem will try to defragment files online, so moving files in the background in order to keep each of them contiguous.... if possible, because this requires enough free space. On the contrary NTFS has been designed as an offline defragmenting system. Microsoft has choosen to not implement an online defragmentation in order to keep a good velocity and so Windows Server pushes a defragmentation process in a scheduled task... for example during the night when employees are sleeping. So keep in mind that UFS, EXTs require a lot of free space to work correctly.
 
Wozzeck.Live said:
You use a swap file.
I personally don't use swap file, I use a dedicated swap partition.
This might be the issue...

When you create a swapfile, it also depends on how you create this swap file.
It is known that swap needs contiguous space.... so if you set a swap file with a maximum but variable size, if your system run out of space, it may have some issues to find contiguous space and swapping process goes to error.
If you have defined a fixed swap size, problem shouldn't occur because system should have written a file filled with zero to reserve the contiguous space

According to me using a swap file is not the good solution
You must set a swap partition that is the best guaranty that contiguous max swap size will be always here
You can possibly add a swap file as a complement to a swap partition
Click to expand...

The swapfile was created like this:

dd if=/dev/zero of=/swapfile.dat bs=1m count=8192
chmod 0600 /swapfile.dat

I added this to /etc/fstab:

md99 none swap sw,file=/swapfile.dat 0 0


I used a dedicated swap partition before but that was only 4GB, the swapfile is 8GB (2 times the memory size).

Actually it all happened after the latest pkg update -f and pkg upgrade I did last week (old hdd with bad sectors, seagate barracuda 7200.7 120GB) and yesterday (another hdd (old but no bad sectors), seagate barracuda 7200.10 320GB)

Will try a clean install later with a dedicated swap partition of 8GB instead of 4GB, but my gut feeling says nvidia/kde.
 
@ rsronin - I agree that a dedicated swap partition is a good thing to have for KDE, and I'd also suggest turning off the File Search feature in your KDE System Settings. The file search is a big CPU and memory hog, and whenever I've left it on, I've experienced core dumps and thrashing problems which I don't experience otherwise. I also go though all the other settings and turn off all the busy features I never use, because frankly KDE is way too busy out of the box IMO.

I've been trying out various variants of kde4, kde5, and plasma5-plasma on a 32-bit Dell with only 3 GB of RAM, but I always have a swap partition just under 6 GB in size according to FreeBSD recommendations. If I partition exactly 6 GB (2 * RAM), I always get complaints in dmesg that it's just a little too big, so I reinstall from scratch and adjust the swap partition size according to their exact recommendation in dmesg. (It always works out to 6110720K in FreeBSD 11.2 and 6096384K in FreeBSD 12.0, don't ask me why.) The resulting configurations run well and smoothly, even if a little slowly due to my limited resources, and even on top of apache and postgresql server services running on the same hardware. No coredumps and no thrashing under reasonable loads, but it definitely does need a little tweaking in the System Settings to avoid them.

Another way of reducing resource usage is to install the plasma5-plasma port or package instead of kde5. You can then add only the features you actually use, like konsole, dolphin, firefox, libreoffice, kcalc, vlc, and/or whatever, and avoid a lot of the overhead that comes with all the built-ins. But my systems still run smoothly even with the full blown kde4 or kde5 software-- but still, only after I disable the baloo file search. They may have improved the file search programs in the newer editions but I wouldn't know since I don't need and always disable it anyway.
 
Vull said:
@ rsronin - I agree that a dedicated swap partition is a good thing to have for KDE, and I'd also suggest turning off the File Search feature in your KDE System Settings. The file search is a big CPU and memory hog, and whenever I've left it on, I've experienced core dumps and thrashing problems which I don't experience otherwise. I also go though all the other settings and turn off all the busy features I never use, because frankly KDE is way too busy out of the box IMO.

I've been trying out various variants of kde4, kde5, and plasma5-plasma on a 32-bit Dell with only 3 GB of RAM, but I always have a swap partition just under 6 GB in size according to FreeBSD recommendations. If I partition exactly 6 GB (2 * RAM), I always get complaints in dmesg that it's just a little too big, so I reinstall from scratch and adjust the swap partition size according to their exact recommendation in dmesg. (It always works out to 6110720K in FreeBSD 11.2 and 6096384K in FreeBSD 12.0, don't ask me why.) The resulting configurations run well and smoothly, even if a little slowly due to my limited resources, and even on top of apache and postgresql server services running on the same hardware. No coredumps and no thrashing under reasonable loads, but it definitely does need a little tweaking in the System Settings to avoid them.

Another way of reducing resource usage is to install the plasma5-plasma port or package instead of kde5. You can then add only the features you actually use, like konsole, dolphin, firefox, libreoffice, kcalc, vlc, and/or whatever, and avoid a lot of the overhead that comes with all the built-ins. But my systems still run smoothly even with the full blown kde4 or kde5 software-- but still, only after I disable the baloo file search. They may have improved the file search programs in the newer editions but I wouldn't know since I don't need and always disable it anyway.
Click to expand...
I experimented both with kde4 and kde5 before. I disabled file searching with balooctl disable on those installations. Also I just installed only plasma5-plasma with some other kde applications like dolphin and konsole.

I just reinstalled FreeBSD-12-RELEASE and gave it 8GB of dedicated swapspace. On purpose I did not upgrade to p3 ( freebsd-update fetch and freebsd-update install). Neither did I use the latest pkg repo, but the quarterly one. For the time being it works now but scared to upgrade.

Code: 
$ cat /etc/fstab
# Device        Mountpoint      FStype  Options Dump    Pass#
/dev/ada2p2     /               ufs     rw      1       1
/dev/ada2p3     none            swap    sw      0       0

$ gpart show
=>       34  488397101  ada0  GPT  (233G)
         34       2014        - free -  (1.0M)
       2048   40960000     1  linux-data  (20G)
   40962048   40960000     2  linux-data  (20G)
   81922048   40960000     3  freebsd-ufs  (20G)
  122882048  348160000     4  linux-data  (166G)
  471042048   17354752     5  linux-swap  (8.3G)
  488396800        335        - free -  (168K)

=>        34  5860533101  ada1  GPT  (2.7T)
          34        2014        - free -  (1.0M)
        2048  5860530176     1  linux-data  (2.7T)
  5860532224         911        - free -  (456K)

=>       40  625142368  ada2  GPT  (298G)
         40       1024     1  freebsd-boot  (512K)
       1064  608174080     2  freebsd-ufs  (290G)
  608175144   16967264     3  freebsd-swap  (8.1G)

=>        40  1953525088  ada3  GPT  (932G)
          40        2008        - free -  (1.0M)
        2048  1953521664     1  freebsd-ufs  (932G)
  1953523712        1416        - free -  (708K)

$ uname -a
FreeBSD n54l 12.0-RELEASE FreeBSD 12.0-RELEASE r341666 GENERIC  amd64

$ cat FreeBSD.conf 
# $FreeBSD: releng/12.0/usr.sbin/pkg/FreeBSD.conf 340161 2018-11-05 21:28:32Z gjb $
#
# To disable this repository, instead of modifying or removing this file,
# create a /usr/local/etc/pkg/repos/FreeBSD.conf file:
#
#   mkdir -p /usr/local/etc/pkg/repos
#   echo "FreeBSD: { enabled: no }" > /usr/local/etc/pkg/repos/FreeBSD.conf
#

FreeBSD: {
  url: "pkg+http://pkg.FreeBSD.org/${ABI}/quarterly",
  mirror_type: "srv",
  signature_type: "fingerprints",
  fingerprints: "/usr/share/keys/pkg",
  enabled: yes
}
$
 
I've been using FreeBSD-12.0-RELEASE-p3, with various plasma5 versions up to and including 5.14.5_1 from both ports and packages, and have had no serious memory usage problems with any of it. Also, since kde4, which seemed to require hald_enable="YES" in /etc/rc.conf, I never enable the HAL daemon for plasma5, but instead, for automounting my USB drives, I use sysutils/automount for packages, or the build option BSDISKS for ports.

On FreeBSD-12.0-RELEASE I have to install sysutils/automount version 1.6.0 or higher, instead of the package version, which is still stuck on version 1.5.8. Automount version 1.6.0 or 1.6.1 can be downloaded and easily installed from here: https://github.com/vermaden/automount

Firefox is my biggest memory user so I monitor it with top, and have to restart it occasionally. Here's my present top summary:
Code: 
last pid: 19356;  load averages:  1.51,  1.56,  1.00     up 1+09:15:48  14:49:51
67 processes:  1 running, 66 sleeping
CPU:  1.2% user,  0.0% nice,  0.0% system,  0.0% interrupt, 98.8% idle
Mem: 410M Active, 1436M Inact, 214M Laundry, 228M Wired, 83M Buf, 702M Free
Swap: 5953M Total, 8944K Used, 5945M Free
 
Just yesterday I was experiencing this issue also. Lots of messages on the console about getswapspace failed. This is on a FreeBSD 12.0-RELEASE-p3 amd64 system, ZFS, everything built from sources (base system, custom kernel, ports, no pkgs) with plasma5-plasma-5.15.0 and nvidia-driver-390.87_2. Strange thing is, when I became aware of the problem, only 1% of the available swap space (4 GiB swap / 16 GiB RAM) was in use and I did not see any ill effects either. I then replaced my tmpfs(5) /tmp filesystem with a ZFS dataset, rebuilt/installed kernel and nvidia-driver and so far I have not encountered the problem again since then. Lets hope it stays that way :rolleyes:
 
I just reinstalled FreeBSD-12-RELEASE and gave it 8GB of dedicated swapspace. On purpose I did not upgrade to p3 (freebsd-update fetch and freebsd-update install). Neither did I use the latest pkg repo, but the quarterly one. For the time being it works now but scared to upgrade.
Click to expand...

So I upgrade to FreeBSD-12.0-RELEASE-p3 and switched to the 'latest' pkg repo. No problem with swapspace , but plasma5/kwin_X11 hangs and I am not able to use the desktop, just a black screen, an empty panel and a mousepointer. The hardware is an HP Microserver n54l with an Nvidia GT710 card. I can not check the other plasma5 installation on my Thinkpad T430 anymore because I had to sent it back for an RMA.
 
rsronin said:
So I upgrade to FreeBSD-12.0-RELEASE-p3 and switched to the 'latest' pkg repo. No problem with swapspace , but plasma5/kwin_X11 hangs and I am not able to use the desktop, just a black screen, an empty panel and a mousepointer. The hardware is an HP Microserver n54l with an Nvidia GT710 card. I can not check the other plasma5 installation on my Thinkpad T430 anymore because I had to sent it back for an RMA.
Click to expand...
Ouch. If you switch your console to ttyv0 with the CTRL-ALT-F1 key combination, are there any interesting messages displayed there? (You should be able to get back to the desktop with CTRL-ALT-F9 as you probably know.) Before regressing back to a previous configuration, you might want to log in on another virtual terminal and inspect your messages with dmesg -a and also check out the contents of the /var/log/Xorg.0.log file.

I'm assuming you have dbus_enable="YES" in your /etc/rc.conf file. Are you using sddm or some other login manager? Before giving up on this configuration, I'd consider disabling the login manager in rc.conf and rebooting ( sddm_enable="NO"). Then you could login as your non-root user and create an .xinitrc file in the non-root user's home directory, with the contents exec ck-launch-session startkde -- and then after that, you could try again to start X by entering the startx command on that non-root user's terminal, and see all the X feedback messages as they occur, in the hope of debugging the problem. It would be better to do this remotely via ssh from another computer with a GUI terminal and scrollback feature, but since your Thinkpad is in the shop you might need to do it from another virtual terminal. Sorry if I led you into this by suggesting that the upgrade wouldn't cause any problems but I really didn't think that it would. And thanks anyway for keeping us up to date on your results.
 
rsronin said:
So I upgrade to FreeBSD-12.0-RELEASE-p3 and switched to the 'latest' pkg repo. No problem with swapspace , but plasma5/kwin_X11 hangs and I am not able to use the desktop, just a black screen, an empty panel and a mousepointer. The hardware is an HP Microserver n54l with an Nvidia GT710 card. I can not check the other plasma5 installation on my Thinkpad T430 anymore because I had to sent it back for an RMA.
Click to expand...
Try reinstalling the nvidia-driver and see if that helps. You might also want to try and build it from ports instead of using the package.
 
Vull said:
Ouch. If you switch your console to ttyv0 with the CTRL-ALT-F1 key combination, are there any interesting messages displayed there? (You should be able to get back to the desktop with CTRL-ALT-F9 as you probably know.) Before regressing back to a previous configuration, you might want to log in on another virtual terminal and inspect your messages with dmesg -a and also check out the contents of the /var/log/Xorg.0.log file.

I'm assuming you have dbus_enable="YES" in your /etc/rc.conf file. Are you using sddm or some other login manager? Before giving up on this configuration, I'd consider disabling the login manager in rc.conf and rebooting ( sddm_enable="NO"). Then you could login as your non-root user and create an .xinitrc file in the non-root user's home directory, with the contents exec ck-launch-session startkde -- and then after that, you could try again to start X by entering the startx command on that non-root user's terminal, and see all the X feedback messages as they occur, in the hope of debugging the problem. It would be better to do this remotely via ssh from another computer with a GUI terminal and scrollback feature, but since your Thinkpad is in the shop you might need to do it from another virtual terminal. Sorry if I led you into this by suggesting that the upgrade wouldn't cause any problems but I really didn't think that it would. And thanks anyway for keeping us up to date on your results.
Click to expand...

Yes, I always have dbus_enable="YES" in /etc/rc.conf file, not using hald_enable and for mounting USB drives I use sysutils/dsbmc/. Last weekend I already had this problem and I tried to ssh into the machine but it was not responding.

In the meantime I did:

sudo freebsd-update fetch
sudo freebsd-udate install

and because I switched to the latest pkg repo:

sudo pkg update -f
sudo pkg upgrade -fy


And yes I was using x11/sddm to login and start the desktop. It resulted in an unresponsive desktop (wallpaper and panel visible this time after a long wait) and a lot of disk activity. I switched to ttyv0 with the CTRL-ALT-F1 and again there were those messages: swap_pager_getswapspace: failed. Also messages that processes were killed (plasmashell, kwin_X11 etc.) I checked with top and only 22mb of 8125mb of swapspace was used.

I will try the startx route with .xinitrc
 
mickey said:
Try reinstalling the nvidia-driver and see if that helps. You might also want to try and build it from ports instead of using the package.
Click to expand...
I would love to go to ports only, last year and my first FreeBSD year, I learned not to mix pkg and ports ;-)
Also wondering how many hours it will take to compile all those packages on an old thinkpad or even worse the hp microserver, apart from me setting up poudriere.
 
You must log in or register to reply here.
Back
Top