1

So we have a large number of scripts running on our server that periodically open up mysql connections and we are generally good about closing the connections as soon as they are done being used.

Lately though we've been seeing massive spikes in connections (upwards of 8000) which crash our server. The outage only lasts a few seconds but it throws a whole bunch of dependent functions into disarray and although manageable is obviously undesirable.

My guess is somewhere a script is running that causes connections to be opened in some kind of loop, possibly a cron.

Problem is I can't seem to find a pattern of when this is happening and I can't capture the ports the connections are eating up since the server crashes milliseconds after the connections begin to open.

What I'm looking for is some strategy or approach I could use to identify where these connections are suddenly coming from.

  • Check your firewall logs? – HTTP500 Nov 16 '15 at 22:42
  • What does the mysql show processlist give you before and after? Can you see the connections with netstat -atnp to the server? Is mysql configured for the load requirements correctly? If the server's max connections are reached, it can cause a backup and mysql chokes. – Gmck Nov 17 '15 at 0:26
0

You can capture the traffic with tcpdump, store it and later analize it.

tcpdump -s 0 -i eth0 port 3306 -w capture.pcap 

And could be a good idea forbid the MySQL traffic and let pass just the authorized clients.

  • I'm going to try this, I'll let you know how it goes – ajax1515 Nov 18 '15 at 15:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.