Solved Massive CPU overuse after update to 5.23

Hi, run 12.3 and performed the quarterly update to Plasma 5.23. I'm now seeing the cores pegged to 90% when I start plasma and they stay there. Also remarkable memory use of 3GB compared to under 1GB before.

I didn't add anything. Any ideas?
 
I've never used KDE / Plasma but there are several people here that do which will potentially be able to help.
I think what would be helpful is if you could post the output of top(1).
 
phalange : Are you running Xorg or Wayland? I'm also assuming you're using packages, rather than ports. Another idea that comes to my mind is: Did you reboot since the upgrade to Plasma 5.23? The idea of trying to avoid a reboot following an upgrade is overrated.
 
freebsd-version -kru ; uname -aKU

cores pegged to 90%

top -m io

– is there excessive use by any process for an extended period?



Here, no recent problem with latest.

Code:
Operating System: FreeBSD 14.0
KDE Plasma Version: 5.23.5
KDE Frameworks Version: 5.90.0
Qt Version: 5.15.2
Kernel Version: 14.0-CURRENT (64-bit)
Graphics Platform: X11
Memory: 15.9 GiB of RAM
Graphics Processor: AMD TURKS

Code:
% freebsd-version -kru ; uname -aKU
14.0-CURRENT
14.0-CURRENT
14.0-CURRENT
FreeBSD mowa219-gjp4-8570p-freebsd 14.0-CURRENT FreeBSD 14.0-CURRENT #1 main-n252531-0ce7909cd0b-dirty: Wed Jan 19 13:29:34 GMT 2022     root@mowa219-gjp4-8570p-freebsd:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG  amd64 1400048 1400048
%

<https://bsd-hardware.info/?computer=6fbb1f806232>
 
Thanks for the replies; abishai was right.

baloo_file_extractor was pegging the CPU use over 90% and hogging gobs of ram, according to top and htop.
I deleted the directory ~./local/share/baloo
restarted
then monitored baloo with balooctl status

It took about an hour to rebuild its database, but now it is running silently and CPU + ram use are as expected.
 
Baloo is a huge pain to KDE Plasma, either in FreeBSD or in Linux, and especially when large disks with millions of files are present. It can make Plasma totally unresponsive, with only the mouse cursor moving without even responding to CTRL+ALT+Fn and alike shortcuts in order to escape the graphic mode. Only a hardware reset or cutting down the power can force that hang to stop.


On a clean install and no additional storage, so a small number of indexed files, baloo does not have a visible impact.


There are 3 main causes that can create a big slowdown or make Plasma unresponsive, apparently a hang:

1. - first indexing when content indexing is enabled - that will enable indexing each word inside files, a massive slowdown and impossible to use in case of millions of files - disable content indexing, either from the GUI or from the baloo's config file

2. - at each restart, baloo will refresh its database. I have a few TB of files on a HDD, and at each restart it takes about 15 minutes of intense crunching the disk's heads. As a workaround I've disabled the baloo monitoring service.

3. - baloo tries to monitor for any changes of all files in it's database at the the same time, live monitoring. It is doing that using kernel features, therefore it will use vfs.numvnodes, and will keep increasing the max vfs nodes limit, which will make Plasma completly unresponsive.


There are many details and tweaks I had to make against baloo and an 8TB storage, too many to retype all these inside this post, so I'll just add my notes regarding baloo on FreeBSD. When file:/// are mentioned in the notes, that is because I sometimes save webpages locally, those pages can be eventually found online by searching for the webpage's title.


All the info is just draft notes, I hope it helps untangle the baloo issues, took me a lot of time to figure them out:

Code:
2022-01-07

- run htop

    - "baloorunner" is the file searcher (correspond to the checkbox "Enable File Search" from GUI - type indexer from krunner to find it inside the System Settings GUI)

        - must be enabled in order to be able to search (files) from the Krunner bar


    - "baloo_file" is the File Indexer

        - must uncheck the box "Also index file content" from GUI

        - best would be to keep the indexer disabled and only run it manually, once in a while


        - if not, each indexed file is also monitored for changes by the kernel

            - this monitoring also reserves memory for each file (i.e. 13 GB! usage of RAM for 2_300_000 files indexed/watched)

            - another issue is that there is a predefined "kern.maxvnodes"

                - the parameter can be read with "sysctl kern.maxvnodes"


                - !!! once the number of files watched exceed this limit,

                  !!! the OS increases it automatically, but only once per second or so.

                  !!!  As a result, the whole OS/kernel will become unusable slow, even the non GUI terminal!!!


                - to display the current number of used vnodes, "sysctl vfs.numvnodes"

                - to increase the max vnodes number (until the next restart?), "sudo sysctl kern.maxvnodes=3000000"

                    - ??? untested (not sure if that's the right place):  permanent increase from /etc/sysctl.conf ???

                    - read also file:///home/aaaa/_FreeBSD/baloo/2022-01-07 - 11.11. Tuning Kernel Limits - people.freebsd.org.html


                - made mine 3 millions because after a finished file indexing of wd8TB (without indexing file content) the vfs.numnodes was more than 2.3 millions and it that was using more than 13 GB of RAM


            - once the indexing is finished, the indexer can be stooped only to re-index manually when necessary, then reboot to regain those 13GB of occupied memory


    - for more info read


            file:///home/aaaa/_FreeBSD/baloo/2022-01-07 - 230726 – sysutils_kf5-baloo_ freezes the system - bugs.freebsd.org.html

            file:///home/aaaa/_FreeBSD/baloo/2022-01-07 - Chapter 12. Configuration and Tuning _ FreeBSD Documentation Portal - docs.freebsd.org.html




- more bloatware to remove suggested:  Akonadi, nepomuk, strigi, baloo ... bloatware


            - file:///home/aaaa/_FreeBSD/baloo/2022-01-07 - Problems with baloo _ The FreeBSD Forums - forums.freebsd.org.html


- suggested baloo alternative:  recoll


        - https://www.freshports.org/deskutils/recoll/

        - https://forums.freebsd.org/threads/baloo_file_extractor-crashes-every-time-how-can-i-disable-it.83103/

        - file:///home/aaaa/_FreeBSD/baloo/2022-01-07 - baloo_file_extractor crashes every time _ how can I disable it _ _ The FreeBSD Forums - forums.freebsd.org.html


- just FYI, baloo FreeBSD port page


    - https://www.freshports.org/sysutils/kf5-baloo/

    - file:///home/aaaa/_FreeBSD/baloo/2022-01-07 - FreshPorts -- sysutils_kf5-baloo_ KF5 Framework for searching and managing user metadata - www.freshports.org.html




aaaa@bsd:~/.local/share/baloo % ls -l

total 90113

-rw-r--r--  1 aaaa  aaaa  206729216 Jan  7 19:07 index

-rw-r--r--  1 aaaa  aaaa       8192 Jan  7 19:03 index-lock


aaaa@bsd:~/.local/share/baloo % balooctl status

Baloo File Indexer is running

Indexer state: Idle

Total files indexed: 470,991 <--- ?????

Files waiting for content indexing: 0

Files failed to index: 0

Current size of index is 197.15 MiB

aaaa@bsd:~/.local/share/baloo %


GUI properties for wd8TB

Size:   5.7 TiB (6,250,088,856,816)

        2,184,724 files, 237,952 sub-folders




- "balooctl disable" will stop "baloo_file" (the file indexer), but KDE will restart it after a reboot (see also file:///home/aaaa/wd8TB/2019/Z/_hobby/_Linux/baloo/autostart disabled crunching disks.txt )




- How to stop indexing to start by itself after a reboot (after a reboot, the indexer will not remember its current state, it will start as dictated by the /usr/local/etc/xdg/autostart/baloo_file.desktop):


        sudo nano /usr/local/etc/xdg/autostart/baloo_file.desktop

            # find the lines

            #   X-KDE-autostart-condition=baloofilerc:Basic Settings:Indexing-Enabled:true

            #   X-GNOME-Autostart-enabled=true

            # change them to false, save and reboot


            X-KDE-autostart-condition=baloofilerc:Basic Settings:Indexing-Enabled:false

            X-GNOME-Autostart-enabled=true




- to manually run the indexer


        sysctl vfs.numvnodes

        sudo sysctl kern.maxvnodes=3000000

        sysctl kern.maxvnodes

        sysctl vfs.numvnodes


        ls -lh /home/aaaa/.local/share/baloo/

        ls -l /home/aaaa/.local/share/baloo/


        balooctl status

        balooctl check


        # wait to finish

        balooctl status


        balooctl disable

        sudo reboot



aaaa@bsd:~ % sysctl vfs.numvnodes

vfs.numvnodes: 2362490

aaaa@bsd:~ % ls -l /home/aaaa/.local/share/baloo/

total 384313

-rw-r--r--  1 aaaa  aaaa  887222272 Jan  7 23:44 index

-rw-r--r--  1 aaaa  aaaa       8192 Jan  7 23:26 index-lock

aaaa@bsd:~ % ls -lh /home/aaaa/.local/share/baloo/

total 384313

-rw-r--r--  1 aaaa  aaaa   846M Jan  7 23:44 index

-rw-r--r--  1 aaaa  aaaa   8.0K Jan  7 23:26 index-lock

aaaa@bsd:~ % balooctl status

Baloo File Indexer is running

Indexer state: Idle

Total files indexed: 2,196,263

Files waiting for content indexing: 0

Files failed to index: 0

Current size of index is 846.12 MiB

aaaa@bsd:~ % balooctl disable

Disabling and stopping the File Indexer

aaaa@bsd:~ % balooctl status

Baloo is currently disabled. To enable, please run balooctl enable

aaaa@bsd:~ % sysctl vfs.numvnodes

vfs.numvnodes: 2362488

aaaa@bsd:~ % ls -l /home/aaaa/.local/share/baloo/

total 384313

-rw-r--r--  1 aaaa  aaaa  887222272 Jan  7 23:44 index

-rw-r--r--  1 aaaa  aaaa       8192 Jan  7 23:47 index-lock

aaaa@bsd:~ % ls -lh /home/aaaa/.local/share/baloo/

total 384313

-rw-r--r--  1 aaaa  aaaa   846M Jan  7 23:44 index

-rw-r--r--  1 aaaa  aaaa   8.0K Jan  7 23:47 index-lock

aaaa@bsd:~ %
 
So baloo is roughly equivalent to Windows "Indexing Service"? One of those things that I've never found useful and pretty much turn off on systems.
Glad the OP found the root cause.
 
So baloo is roughly equivalent to Windows "Indexing Service"? One of those things that I've never found useful and pretty much turn off on systems.
Glad the OP found the root cause.
It's worse, because afaik, the windows indexing use an specific kind of database for that and it's the only indexer, while in kde you have baloo and there's also the other indexer (I forgot the name, the one the start menu uses isn't tied to baloo) and if you use kmail and friends, you'll also have akonadi with his own kind of indexing.
 
  • Like
Reactions: mer
hmm... /bin/find and /usr/libexec/locate.updatedb seem to be ignored in this conversation. Also, shouldn't it be possible to limit Baloo indexing to specific locations? Like, tell it to NOT index that 8TB storage.
 
It is possible to exclude a specified location from baloo, but then I'll need something else to index by name the 3 million files in the 8TB storage.
- find would be too slow, a search will take many minutes and stress the HDD heads at each search
- locate is OK in terms of speed, except it is indexing files only, and it is not integrated in the KDE Plasma GUI, so good only from a terminal and with more typing (this FreeBSD w 8TB attached is used mostly as a GUI desktop)

My typical need with baloo is to fast-open files/applications/webtabs/etc. by hitting the SUPER key + typing 3-4 letters from something I want to open. That would automatically open a small menu in the kluncher, menu populated with the most accessed items containing the letters I typed, usually the one I need is the top, so I just hit ENTER, then the rest of file associations will kick in, and open my file with the associated program. Very similar to what TAB autocompletion does in a terminal, except baloo is for the GUI. Has many other features but I don't need those.

Without baloo, the GUI Klauncher will only be populated with programs but no files. I'll open another thread to search for a baloo replacement.

My reply here was just for the docs. I've noticed this thread was already marked as solved.
 
Thanks for the replies; abishai was right.

baloo_file_extractor was pegging the CPU use over 90% and hogging gobs of ram, according to top and htop.
I deleted the directory ~./local/share/baloo
restarted
then monitored baloo with balooctl status

It took about an hour to rebuild its database, but now it is running silently and CPU + ram use are as expected.

If you want, you also can disable it with

balooctl disable
 
Back
Top