1

Our organization has been having a reoccurring problem at least once a week where our website gradually slows down until it stops responding to requests, can't SSH in, can't connect to websites, can't connect directly to MySQL database. This has been occurring for months and still haven't found the cause. We left the top command running on our webserver (CentOS 6) and connected over VMWare remote console once the server started slowing down to see if there was a specific process causing the issue (screenshot attached below).

The only other clue we noticed is that when the server starts slowing down, if we use the monitoring tools provided with VMWare ESXI we can see the disk read/write spike alot higher than average and stay really high. Only solution we've found is if we're quick enough before the server is going too slow, restart apache or mysql resolves the issue, or just rebooting the webserver. Anyone have any thoughts on possible causes?

top

migrated from webmasters.stackexchange.com Apr 30 '18 at 15:51

This question came from our site for pro webmasters.

  • side note: a monitoring tool like munin-monitoring.org can be useful – Fabian Apr 30 '18 at 16:10
  • Rather than relying on catching something at the right time in top ; please install one of the many more advanced monitoring software packages , or something like new relic that will record trends and performance counters that will allow you to do a proper analyses – HBruijn Apr 30 '18 at 16:23
  • What does LSOF say during these periods? – TheFiddlerWins Apr 30 '18 at 17:18
  • @Fabian server has been up for a little longer than 24 hours and has yet to crash again (although I know it will crash again in a few days), here is a screenshot of my swap usage right now. i.imgur.com/Iywrp9V.png – mark-rodgers May 1 '18 at 23:32
  • @TheFiddlerWins I'll get back to you on this. Next time the server starts to slowdown I'll try to run lsof before it slows down to an unusable state. – mark-rodgers May 1 '18 at 23:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.