Huge failure in vmstat -z result, and thousand of socket connections

Hi All,

I just got a weird issue on my web server. Sometimes the MySQL will take up 2000% wcpu resource, and there are almost 1000 query waiting in the MySQL queue.

At this time, if I run netstat -an, I can see that there are almost 20,000 socket connections on my server. When MySQL is running well, it's only less than 100 socket connections.

(On my server php-fpm, mysql, memcached are all using socket connections.)

And another thing I found is that for vmstat -z result, there are lots of failures as below:

Code:
ITEM  SIZE  LIMIT  USED  FREE  REQ FAIL SLEEP
syncache:  160,  15375,  19,  5831,2445678425,455673534,  0
64 Bucket:  512,  0,  740,  2028,1061219278,306675838,  0
128 Bucket:  1024,  0,  765,  971,593895772,33879138,  0
256 Bucket:  2048,  0,  1051,  223,1426185690,607523970,  0
vmem btag:  56,  0, 1417610, 1363957,905427542,26092,  0


I'm not sure that these failures are related with my problem, so I just put them there.

Can anyone help?
 
Last edited by a moderator:
I think you need to take a look at your MySQL server, find out why queries are taking so long. Apparently they take too long and queries have started to queue up. Enable the slow-query log and check that. Look at the queries themselves, could they be optimized? Perhaps you need to create indexes? The slow-query log should be helpful with that.
 
Back
Top