My server's CPU has been spiking out for the past few days, to the point that it drops requests. I've been looking through the logs, and see certain IP's in the logs that are just simply downloading the content of my site, such as the .js, .css. image files, etc.. over and over again. The IP is usually based in China. I also see other requests from them trying to find files on the server that don't exist. Like
These addresses have nothing to do with our site.
I tried blocking the first IP, but it looks like its now coming through many different IPs. I think this is what's making my CPU spike.
My question is, how do I prevent this type of stuff from happening? How can I respond to keep my site running and available?
I was reading how there are certain ways to block people who are sending too many requests for a certain time period, but I don't want to accidentally block things like BingBot, GoogleBot and other spiders out there as well. What do people do in this case to prevent this type of attack? If it is an attack?