Updated a bunch of servers, as usual quite uneventful. Except for our HAProxy, CPU load slowly crept up and went through the roof after a couple of minutes of running (1600% CPU load ). At first we though we had another DoS (this happens from time to time). But traffic wasn't much, not more than usual any way. Requests looked fine too. Hmm, I did upgrade HAProxy just before the CPU started rising.
Seems there are some bugs in 2.9.1: https://www.haproxy.org/bugs/bugs-2.9.html
Not sure which one exactly was causing the problems but we quickly reverted back to 2.8.5 and the issue disappeared like snow in the sun. We're going to wait with upgrading for 2.9.2 to be released (not available upstream yet)
So, if you are running HAProxy, maybe hold off the update. Or if you want to test it yourself, make sure you have a copy of the old version at hand.
Seems there are some bugs in 2.9.1: https://www.haproxy.org/bugs/bugs-2.9.html
Not sure which one exactly was causing the problems but we quickly reverted back to 2.8.5 and the issue disappeared like snow in the sun. We're going to wait with upgrading for 2.9.2 to be released (not available upstream yet)
So, if you are running HAProxy, maybe hold off the update. Or if you want to test it yourself, make sure you have a copy of the old version at hand.