• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

Solved Great UI/X performance gains by turning off swap

achix

Well-Known Member

Thanks: 36
Messages: 273

#1
Hello,
my workstation with 16GB, running eclipse, jboss, postgresql, KDE, thunderbird, and two bhyve linux guests (one running skype and the other pgbouncer) was having performance problems. Viewing/deleting messages in thunderbird took 2-3 seconds and getting worse, opening web pages on firefox was slow, the system was slow. There was constant disk activity and swaping. Most of the swap (4GB) was used. So I though of buying more more RAM, I added another 16GB, totaling to 32GB and the system at first seemed OK, but later started to behave slowish again. Again there was swapping, and constant disk activity, I could hear the disk every time I clicked on a email. The Inactive memory was always more than 20GB!!!
So I turned off swap. I had tried this before with the 16GB but failed. This time worked. And one day after the system seems and is much more snappier.

So, what's the deal with swap? And free memory , Inactive memory? Lots of oldish posts about it, few solid answers.
I have been running FreeBSD since version 2 IIRC, now I am on 11.1-RELEASE.
 

hotaronohanako

New Member

Thanks: 7
Messages: 17

#2
freebsd in general has a bad performans with swap related stuff you will not fine any solid information about that ! I used to have those problems with swap active its been at least 6 years
 

ronaldlees

Aspiring Daemon

Thanks: 190
Messages: 557

#3
It seems that the system has default values for some memory related tweakables:

https://unix.stackexchange.com/questions/145909/how-much-ram-needed-for-preventing-use-swap

https://wiki.freebsd.org/SystemTuning

Probably you could set one of the values indicated on the linked page(s), and be able to run with swap=on, without ever actually swapping to disk. OTOH, do you need to swap at all? Will you ever need more than 32 GB? If memory is exhausted, you may crash, so it depends how heavy or critical your machine usage is and what your crash tolerance is. I never run any swap on my personal machines. I suppose a server with many clients would be different.

This is a nice question for a FreeBSD VM expert to answer, though. Wading thru the stuff in the second link will give your eyes a glazed look. Probably it would take some experimentation to determine the needed tweaks - or better yet we'd just get an expert's 30 second setup detail. Just turning swap off is easy but crude. I'm sure there are server admins here who've had to eek out that last bit of performance, and know the settings by heart.

Are you using the latest skype in your bhyve? Which Linux? I have always used a Linux physical machine for my skype. What's the "30 second setup detail?"
 

achix

Well-Known Member

Thanks: 36
Messages: 273

#4
Before I turned off swap, I had tried shutting down the bhyve VMs, jboss, eclipse anything that could eliminate out some of the possibly "exotic" factors. (which made no sense since there were still huge amounts of Inactive mem but anyways). Nothing good out of it. Web browsing still crawled. Email viewing crawled. It seemed like the system was trying to go thrashing (a computer term back from the 80s) by any means. IMHO there should be some guidance/docs/tweaks for workstation usage, besides turning off swap. Can't comment on how good this is for servers (heck my workstation acts as a server as well), but for plain usage this sucks. I mean 32GB of RAM, still having lots of unused memory while listening to the disk spinning at every click is no fun.
 

ronaldlees

Aspiring Daemon

Thanks: 190
Messages: 557

#5
Seems like you're using 64 bit FreeBSD, right?
- Nevermind, your bhyve means your running that.

Looks like they took out a sysctl that may have been of help to you, (removed in version 11.1):

https://forums.freebsd.org/threads/64198/

The sysctl was vm.defer_swapspace_pageouts but was pulled in version 11.1, (explanation attached in the linked thread). Since they put the new code described in the explanation into the source recently, maybe there's a bug in it that you could report.

sysctl vm.defer_swapspace_pageouts=1
 

Snurg

Aspiring Daemon

Thanks: 253
Messages: 706

#6
Good tip! Was wondering what caused all these annoying latencies, too.
Turned off swap, system became more responsive INSTANTLY.
Hesitated for long doing this because people said it's bad to disable swap. Doing so seems to be kind of blasphemy.

With >20GB RAM free it's just stupid to swap things out. This is Windows style. (BTW Windows profits a lot, too, if one creates small RAMdisk and uses that for swap, as it insists of having swap...)
Let's see whether this causes problems on FreeBSD....
 

ronaldlees

Aspiring Daemon

Thanks: 190
Messages: 557

#7
Good tip! Was wondering what caused all these annoying latencies, too.
Turned off swap, system became more responsive INSTANTLY.
Hesitated for long doing this because people said it's bad to disable swap. Doing so seems to be kind of blasphemy.

With >20GB RAM free it's just stupid to swap things out. This is Windows style. (BTW Windows profits a lot, too, if one creates small RAMdisk and uses that for swap, as it insists of having swap...)
Let's see whether this causes problems on FreeBSD....
Well, the reason they say that you should have swap space is because if you run out of memory and have no fallback swap, your system could crash (or become unresponsive to the point that you think it has crashed). Been there, done that. But if my system crashes while I'm visiting randomnews.com, I don't care that much. With the memory I have it's never crashed. Could tho.

Another benefit of having swap space is that idle process mem can be swapped out, which may free up memory for the processes you are actually using, and keep them from swapping. I think the idle time can be changed with a sysctl value. So, anyway, sometimes swap is a good thing. The system really should work with swap on, and we shouldn't need to tweak it.

I wonder now if there's a bug that should be looked at in version 11.1, introduced maybe with this VM change. Snurg: Are you running version FreeBSD 11.1?
 

Snurg

Aspiring Daemon

Thanks: 253
Messages: 706

#8
Yes 11.1.
Base system rev 321309 (version of release DVD, just for testing, will svn and build the most recent version in the next days.)

And the change after swapoffing is remarkable.
I was annoyed for quite a while why it looked like when I switched to another virtual screen, there was a delay until the windows were refreshed.
Only a fraction of a second, but very noticeable and, as said, really annoying.

Now refreshing is barely noticeable anymore if at all.
 

dlegrand

Active Member

Thanks: 42
Messages: 148

#12
I am currently using ZFS, having Virtualbox running Windows 10 pro with Edge open, while surfing the Web with Firefox 58 and editing a photo with Gimp.
I usually build my own packages with ports-mgmt/poudriere on my desktop computer, so I had to limit memory usage of ZFS by adding the following line into /boot/loader.conf:
Code:
vfs.zfs.arc_min="1G"
vfs.zfs.arc_max="4G"
I only have 8Mb of memory and a hard drive of 2Tb.
Code:
last pid:  1747;  load averages:  0.52,  0.72,  0.64                                                 up 0+04:27:15  12:15:23
97 processes:  1 running, 96 sleeping
CPU:  0.8% user,  0.0% nice,  6.8% system,  0.2% interrupt, 92.3% idle
Mem: 1206M Active, 1019M Inact, 105M Laundry, 5431M Wired, 142M Free
ARC: 2607M Total, 974M MFU, 1548M MRU, 176K Anon, 16M Header, 69M Other
     2240M Compressed, 3960M Uncompressed, 1.77:1 Ratio
Swap: 12G Total, 12G Free

  PID USERNAME      THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND
 1680 didier         30  20    0  2734M  2309M select  4  10:55  41.66% VirtualBox
  882 root            1  21    0  6365M   113M select  2  13:50   1.97% Xorg
 1713 didier         33  20    0  1879M   397M select  5   0:19   0.64% firefox
 1721 didier         34  20    0  1723M   296M select  4   0:55   0.47% firefox
 1679 didier         15  20    0   138M 28772K select  2   0:05   0.32% VBoxSVC
  443 root            1  20    0 12720K  1588K select  4   0:15   0.32% moused
 1082 didier         59  20    0  2243M   649M select  3   9:51   0.23% firefox
 1674 didier          8  20    0   425M 69696K select  0   0:04   0.19% VirtualBox
 1033 didier          1  20    0   204M 26484K select  4   0:14   0.17% xfwm4
 1677 didier          1  20    0 84008K 11556K select  2   0:03   0.17% VBoxXPCOMIPCD
 1087 didier          4  20    0   329M 64088K select  1   0:54   0.16% python2.7
 1696 didier          1  20    0 20164K  4228K CPU1    1   0:01   0.06% top
 1693 didier          1  20    0 73452K 12220K select  4   0:00   0.03% xterm
  884 mpd             5  20    0   242M 21608K select  5   0:09   0.01% musicpd
  833 root            1  20    0 10472K  1628K select  0   0:02   0.01% powerd
 1051 didier          1  20    0   185M 21596K select  0   0:01   0.01% wrapper-1.0
 1065 didier          4  20    0 79988K  7852K select  3   0:01   0.01% gvfsd-trash
 1076 didier          4  20    0   311M 52396K select  5   0:02   0.01% emacs-25.3
  830 root            1  20    0 20568K 12476K select  4   0:01   0.00% ntpd
 1032 didier          1  20    0 33904K  3540K select  0   0:00   0.00% gpg-agent
  485 _pflogd         1  20    0 10740K  1868K bpf     4   0:00   0.00% pflogd
  921 root            1  20    0 75972K  4872K select  1   0:00   0.00% httpd
 1328 root            1  20    0 23820K  5344K bpf     4   0:00   0.00% tcpdump
 1390 didier         34  20    0  1987M   428M select  1   4:23   0.00% firefox
  790 root           48  52    0  8348K  2316K rpcsvc  0   0:00   0.00% nfsd
 1387 didier         33  20    0  1669M   232M select  2   0:17   0.00% firefox
 1034 didier          4  20    0   260M 29868K select  0   0:06   0.00% xfce4-panel
 1038 didier          4  20    0   246M 33328K select  2   0:06   0.00% xfdesktop
 1684 didier         10  21    0   396M   107M select  3   0:05   0.00% gimp-2.8
  906 didier          1  20    0 55152K  8108K select  3   0:03   0.00% fetchmail
 1068 didier          3  20    0   246M 31672K select  5   0:03   0.00% orage
 1021 didier          2  20    0   210M 22180K select  2   0:02   0.00% xfce4-session
 1069 didier          1  20    0 52364K  5284K select  3   0:02   0.00% xscreensaver
 1042 didier          3  20    0   386M 46908K select  2   0:01   0.00% pidgin
 1063 didier          3  20    0 69376K  6960K select  3   0:01   0.00% at-spi2-registryd
 1036 didier          4  20    0   226M 23076K select  0   0:01   0.00% xfsettingsd
 1025 didier          1  20    0 21636K  3868K select  5   0:01   0.00% dbus-daemon
 

achix

Well-Known Member

Thanks: 36
Messages: 273

#13
After setting kern.ipc.shm_use_phys to default (0) and re-enabling swap, the system seems to not use any swap space, which looks good. Will keep an eye on this.
 

ronaldlees

Aspiring Daemon

Thanks: 190
Messages: 557

#14
From the second link (above):

The kern.ipc.shm_use_phys sysctl defaults to 0 (off) and may be set to 0 (off) or 1 (on). Setting this parameter to 1 will cause all System V shared memory segments to be mapped to unpageable physical RAM. This feature only has an effect if you are either (A) mapping small amounts of shared memory across many (hundreds) of processes, or (B) mapping large amounts of shared memory across any number of processes. This feature allows the kernel to remove a great deal of internal memory management page-tracking overhead at the cost of wiring the shared memory into core, making it unswappable.
 

Snurg

Aspiring Daemon

Thanks: 253
Messages: 706

#15
Seems to be an incredibly bad idea when an OS has default settings to swap out shared memory when there are dozens of GB RAM free.
Anyway, I had turned off shm usage by KDE/Qt by setting QT_X11_NO_MITSHM=1 to avoid this issue.

There is a diagnostic tool that says it can show which processes use how much shared memory, and it was displaying no shared memory being used (dunno whether it was correct, it is a Linux script).

So this crazy issue appears probably not caused by SHM swapping...
 

Eric A. Borisch

Well-Known Member

Thanks: 162
Messages: 262

#16
After setting kern.ipc.shm_use_phys to default (0) and re-enabling swap, the system seems to not use any swap space, which looks good. Will keep an eye on this.
I commented on the other thread, but just to highlight this: going back to the default settings gave you good performance, is that correct? Do you have any other values changed from default via sysctl.conf or loader.conf? Do you know why kern.ipc.shm_use_phys had been set to a non-default value? (Had some port recommended it?)

Thanks!
 

Nicola Mingotti

Active Member

Thanks: 48
Messages: 101

#17
I have an idea, but I am newbie in FreeBSD, I would like to have your opinions before "trashing" my system and loose data.

This is my case. I have a very small VM running FreeBSD-11.1. 5GB-RAM, 20GB-HD, 1GB-Swap, 1 core. It runs in VMWare, OSX host, the metal is a 2009 Macbook Pro. 2 cores, 8GB RAM, 512 GB HD solid state.

-] I use the machine for programming, Web, Mail etc. it is not a server.

-] The machine was slow, but let's say almost as slow as a previous Debian. Ok, I accept it, if I must, it also very small. With a difference respect to Debian, at some point this FreeBSD machine runs out of SWAP and locks me out. Sometimes I can enter a non graphical login console and kill Firefox and company, but not always, let's say 50% of times. When I can't I need to reboot (I don't like it, it remind me Windows and the old MacOS of iMacs ;)

-] A few hours ago i have red a messege here on the forum, and consequently i turned of the swap off !
fantastic ! The sistem now is almost as fast os the OSX running on metal ! I am so enthusiastic I have come back to office to write this message on Friday night:)

-] I checked "kern.ipc.shm_use_phys", mine is set to zero. And was set to zero before, when the machine was very slow.

-] I checked ronaldlees links, but in the first they are talking about "swappiness", there is not such toggle in my "sysctl -a", it is Linux conf. In the second I can't find the key point (I red it fast, maybe i missed it)

-] Snurg suggested to tune ZFS parameters, I will try to understand that way as well. To be true, I was not aware i was not into ext3-4 ;)
And /etc/fstab says I am using UFS.

I was considering another way I could keep swap-off without risk to crash and loose data. And I figured out this option.
1) Consider I have 2 human users in my system "root" and "foo". It is always me.
2) Consider I do almost all work as "foo", I keep only one console open for "root" and i rarely use it (a bounch of times per day, update locate, install packages etc.)
=> What if I set a constraint on memory useage for user "foo", let's say I set a limit for "foo" to use maximum "4GB". In this case the system would always have about "1GB" free, so all root processes should in principle always have rooom enough to allocate their memory. And the system should not crash.

What do you think, is this just fantasy ? May it work ?

Or also, could I set user "foo" not to use the swap and user "root" to do it ?
 

achix

Well-Known Member

Thanks: 36
Messages: 273

#18
I commented on the other thread, but just to highlight this: going back to the default settings gave you good performance, is that correct? Do you have any other values changed from default via sysctl.conf or loader.conf? Do you know why kern.ipc.shm_use_phys had been set to a non-default value? (Had some port recommended it?)

Thanks!
Current settings :
Code:
postgres@smadev:~> cat /boot/loader.conf
#radeon_load="YES"
#nvidia_load="YES"
nvidia-modeset_load="YES"
snd_hda_load="YES"
geom_mirror_load="YES"
#vboxdrv_load="YES"
atapicam_load="YES"

kern.ipc.semmns=600
kern.ipc.semmni=100
postgres@smadev:~>
Code:
postgres@smadev:~> cat /etc/sysctl.conf | egrep -v -e '^#|^$'
hw.snd.default_unit=4
kern.ipc.shmmax=20000000000
kern.ipc.shmall=81920000000
net.inet.ip.forwarding=1
kern.maxfiles=123280
kern.maxfilesperproc=110950
kern.ipc.shm_allow_removed=1
postgres@smadev:~>
nothing spectacular I guess. The kern.ipc.shm_use_phys=1 was supposed to help postgresql run faster, but now PostgreSQL went mmap, so it does not matter anyways.
 

achix

Well-Known Member

Thanks: 36
Messages: 273

#19
Hmm, every night cron does a massive rsync backup, this might have caused some swap usage, but currently top shows :
Code:
last pid: 93510;  load averages:  0.13,  0.27,  0.26   up 30+23:11:09  10:49:15
280 processes: 1 running, 279 sleeping
CPU:  0.3% user,  0.0% nice,  0.6% system,  0.1% interrupt, 99.0% idle
Mem: 1702M Active, 15G Inact, 12G Laundry, 2590M Wired, 690M Buf, 463M Free
Swap: 8000M Total, 1694M Used, 6306M Free, 21% Inuse

  PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND
69283 achill        2  20    0   562M 97972K select  3  65:38   1.41% kdeinit4
70321 achill       45  20    0  2576M   771M select  1 180:07   1.23% firefox
70396 achill       46  20    0  2449M   591M select  2  66:24   0.57% firefox
15260 jboss       131  52    0  3082M   986M uwait   1  26:20   0.55% java
69667 root         12  20    0  1058M   872M kqread  1  44:39   0.51% bhyve
70327 achill       36  21    0  2443M   692M select  0 111:38   0.48% firefox
70883 achill       45  20    0  2395M   534M select  4  74:54   0.36% firefox
70318 achill       61  20    0  2597M   849M select  0 202:36   0.33% firefox
93510 postgres      1  20    0 20164K  4228K CPU3    3   0:00   0.28% top
68230 achill        1  20    0  6438M   165M select  2 156:40   0.27% Xorg
68291 achill        4  20    0   962M   149M select  3  70:48   0.22% kwin
69971 root         14  20    0  5155M   780M kqread  3  15:47   0.21% bhyve
15G Inact, + 12G Laundry and still using swap?? As said in the other brother thread, will test it on job on Monday and the upcoming days, see how it goes and get back to report.
 

PacketMan

Aspiring Daemon

Thanks: 108
Messages: 786

#21
Seems to be an incredibly bad idea when an OS has default settings to swap out shared memory when there are dozens of GB RAM free.
Naaa I disagree there. When architecting something, whether it be an OS, a very very large WAN, a physical structure (house, bridge, etc), a vehicle (car, boat, train, etc) the architects have to take into consideration a lot of input factors and use-case scenarios. In some ways architecting and engineering are about building the perfect compromise. The FreeBSD builders have given you knobs that you can tweak to suit your requirements, but to say the default settings are a bad idea, naaaaaa, there is a much bigger picture involved that's all.

I might tune my settings someday, for a learning experience more than anything, but I do so at my own risk. Also, I don't know if this applies to computing systems, but in large networks we router guys have to work with something known as traffic microbursts. Its out of scope to explain it in this thread, but I am curious to learn if ram microbursts exist, and if so then it would seem having an OS that shuffles off some stuff into swap, just to be ready for a microburst would seem real smart to me.