Was killed: a thread waited too long to allocate a page (MariaDB and Java)

Aknot

Aknot

Hello,

Tonight was the second time databases/mariadb-server and Java (textproc/opensearch) was killed, and I can't figure out why?
I have about 15Gb hard drive space left, and about 1/4 RAM left when the server is working hard.

Could it be a memory leakage using up the RAM hastily?
The FreeBSD 13.2-RELEASE-p4 server is running on a VM Ware VPS.

/var/log/messages
Code: 
Oct 12 03:05:40 srv06 kernel: pid 65135 (java), jid 0, uid 855, was killed: a thread waited too long to allocate a page
Oct 12 03:06:58 srv06 kernel: pid 16625 (mariadbd), jid 0, uid 88, was killed: a thread waited too long to allocate a page

/var/log/dmesg.today
Code: 
pid 65135 (java), jid 0, uid 855, was killed: a thread waited too long to allocate a page
pid 16625 (mariadbd), jid 0, uid 88, was killed: a thread waited too long to allocate a page

/var/log/mysql/mysqld.err
No errors, just logging a startup after spawning shortly after.

/var/log/opensearch/opensearch.log
No errors, didn't respawn, killed and dead.

This is over my head,
Thanks,
 
How much memory does the system have and how much swap has been configured? And are you using ZFS?
 
Yeah, ZFS + Java + MySQL/MariaDB can easily eat up all your memory. Limit ARC on ZFS, and run something like databases/mysqltuner to make sure your pools, buffers and whatnot aren't using more memory than the machine actually has. For your Java processes you typically also want to set -Xmx (max heap size).
 
SirDice => We only have 4Gb of RAM, and that's working fine, mostly. It's a low traffic FAMP-server (500 - 1000 visits a day). When compiling ports, it's a bit low, and that's why we have 9Gb of swap space for these short moments. We have mysqltuner and UFS. I have set -Xmx and -Xme to lower value (don't need 1G).
I have made several changes now like FPM fm=ondemand, increasing kern.maxfiles files (again), tuned MariaDB, Java and Opensearch ( tempfs ) a bit.

cracauer@ Yes it seems, we had a brute force attack, eating all the RAM, that's why this error was sporadic.

Tuning isn't always my cup of tea ;)
Hoping for the best and keeping my eyes at the log files :beer:
 
MySQL/MariaDB does have corner cases where it seems to chew RAM and go into swap if you allocate it too much memory (seemed to be worse under FreeBSD 12) but you might be surprised how even settings that seem conservative give it enough room to cause problems. Good luck!
 
You must log in or register to reply here.
Back
Top