0

I´m using:

  • Google Cloud Compute Engine
  • Linux Debian + Apache2
  • Google Cloud SQL
  • Wordpress

Yesterday my CPU started increasing from 1-5% to 80-90%. I was developing on a wordpress site as it happened, but that could cause the failure. I had the same thing a couple of days ago, so I finally stopped the VM-Instance, cloned it to a new one and also recovered the Database from the day before.

After that it works better than before! Before I had constant 10% CPU usage - after that 1% sometimes up to 5%. Then at 10:10 pm it starts to increase again. Also the pageload increase to 15 Sek and more.

My SQL Instance takes a snapshot everyday between 10:00 pm - 2 am. which is also the time both happened.

Here is a screenshot of the CPU Usage: http://i.stack.imgur.com/TwM1w.png

Maybe the proccesses are helpfull too:

www-data  4184  6.4  1.0 320160 81604 ?        S    13:25   0:38 /usr/sbin/apache2 -k start
www-data  4204  6.0  1.2 332204 93596 ?        S    13:26   0:30 /usr/sbin/apache2 -k start
www-data  4222  6.0  1.2 331948 93340 ?        S    13:28   0:25 /usr/sbin/apache2 -k start
www-data  4244  6.4  1.0 320928 82320 ?        S    13:29   0:21 /usr/sbin/apache2 -k start
www-data  4245  6.4  1.7 373280 134724 ?       S    13:29   0:21 /usr/sbin/apache2 -k start
www-data  4249  6.6  1.2 331696 93136 ?        S    13:29   0:21 /usr/sbin/apache2 -k start
www-data  4253  6.4  1.2 332212 93636 ?        S    13:30   0:16 /usr/sbin/apache2 -k start
www-data  4271  6.2  1.2 332972 94344 ?        S    13:31   0:14 /usr/sbin/apache2 -k start
www-data  4277  6.2  1.5 357044 117944 ?       S    13:32   0:09 /usr/sbin/apache2 -k start
www-data  4278  6.3  1.2 332972 94344 ?        S    13:32   0:09 /usr/sbin/apache2 -k start
www-data  4283  6.3  1.5 356788 117664 ?       S    13:32   0:09 /usr/sbin/apache2 -k start
www-data  4286  5.8  1.2 331696 92852 ?        S    13:32   0:08 /usr/sbin/apache2 -k start
www-data  4287  6.6  1.5 357040 117576 ?       S    13:32   0:09 /usr/sbin/apache2 -k start
www-data  4300  5.7  1.2 331180 92316 ?        S    13:34   0:03 /usr/sbin/apache2 -k start
www-data  4301  6.0  1.2 332976 94120 ?        S    13:34   0:03 /usr/sbin/apache2 -k start
www-data  4302  5.9  1.2 332976 94116 ?        S    13:34   0:03 /usr/sbin/apache2 -k start
www-data  4304  6.2  1.2 331180 92316 ?        S    13:34   0:03 /usr/sbin/apache2 -k start
www-data  4305  5.8  1.2 333232 94392 ?        S    13:34   0:03 /usr/sbin/apache2 -k start
www-data  4306  6.2  1.2 333232 94372 ?        S    13:34   0:03 /usr/sbin/apache2 -k start

I´m not a pro, so every Litte Help would be great!

  • I would suspect mysql. It can get very hungry in memory if not well configured. In my.cnf, turn on slow queries logging to check if your application codebase can be improved. Also, try using php-fpm instead of mod_php , along with apache 2.4 instead of apache 2.2. much more efficient resourcewise. – pixeline Feb 5 '15 at 14:46
  • If your Wordpress is not public it is very strange you have 2700 writes/s. Verify you are not getting external connection with netstat command or close public access with a firewall rule if it is for development only. Check the Wordpress plugins you installed. If it happens again it will be easier to troubleshoot when server is under unwanted load. Enable SQL query log to see what is being written to DB. – Paolo P. Feb 6 '15 at 7:43
  • MySQL was my thought too but as it is a cloud service from google it´s not running on that server. The Performance on the Cloud SQL Maschine is fine. btw. it´s not a machine that i can configure like a VM Instance. – MCG Feb 9 '15 at 13:45
  • Any update? How many connections are being handled actually ? If you want to troubleshoot queries on Cloud SQL you can temporarily enable logging as documented at cloud.google.com/sql/faq#transactionlogs – Paolo P. Feb 11 '15 at 12:01
  • Actually it´s working fine again. Sometimes it still slow but not that bad anymore. So, it´s still not fine but i´ll comeback later to this Problem. I´ll come back to that Problem when i´m done with some other things. Thanks so far! .. Also for the link. I´ll check that as next step. – MCG Feb 17 '15 at 0:17
1

I'm not 100% sure of your issue. You stated that the snapshot occurs at the time of the spikes, so it seems that you have your culprit. If this is a VM snapshot then you are going to see the host CPU usage spike as snapshots do take some time to complete, and depending on your resources available on the host, this will affect the guest VMs.

If you are saying the DB snapshot takes place at that time, then you are going to see spikes for this as well. The DB has to lock processes and tables in order to do a mysqldump of the DB. I don't know the size of your DB either but here's a good write-up of optimizing your snapshot processes for MySQL: https://dba.stackexchange.com/questions/20/how-can-i-optimize-a-mysqldump-of-a-large-database/2227#2227

Are you seeing a lot of swap (pagefile) space being used during the high CPU usage on you host / VM? This could indicate a problem with your VM's resource management settings. CPU usage isn't directly related to memory use, but will be impacted in high swapping events, which is very likely with a poorly configured DB VM. One of the biggest mistakes I've seen with peoples' DB server setups is lack of spindles and lack of memory allocated.

I again don't have all of the info on this, but you should also check that you've allocated enough CPU resources to the DB VM to ensure proper functionality.

You should be very careful with restoring MySQL DBs as well. You will lose all updates to the database after the point you chose to restore from, and often times need to resync the DB after the restore...not fun.

  • thanks for your answer! actually my database is 350 MB. i think that´s should be fine, or? – MCG Feb 5 '15 at 14:33
  • hit enter to fast.. I didn´t do a snapshot during that time. It was just the Database that made a automatic backup. But maybe it´s not because of that. I´m jsut wonderin why their are so many apache processes with so much CPU Usage? And why it´s starting from nearly nothing. Even a Shutdown did change anything. When i restart all the processes are the same as before even without visiting the website. Do you know any debug i can do so i can see what is happeneing? – MCG Feb 5 '15 at 14:38
  • I also saw that my SQL Write Request start to increase 10 hours before. 2700 Writes per Second. – MCG Feb 5 '15 at 14:44
  • 1
    You could check the /etc/apache/log/ (<-yours) or /etc/httpd/log file depending on your flavor of Linux. As for debugging MySQL here's a good short resource: dev.mysql.com/doc/refman/5.0/en/debugging-server.html. You can also dump the locks to the MySQL logs with those commands which could show possible issues. Find your MySQL log location by looking in /etc/mysql/my.cnf. – cclater Feb 5 '15 at 16:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.