How can I prevent a DDOS attack on Amazon EC2?

Question

One of the servers I use is hosted on the Amazon EC2 cloud. Every few months we appear to have a DDOS attack on this sever. This slows the server down incredibly. After around 30 minutes, and sometimes a reboot later, everything is back to normal.

Amazon has security groups and firewall, but what else should I have in place on an EC2 server to mitigate or prevent an attack?

From similar questions I've learned:

• Limit the rate of requests/minute (or seconds) from a particular IP address via something like IP tables (or maybe UFW?)
• Have enough resources to survive such an attack - or -
• Possibly build the web application so it is elastic / has an elastic load balancer and can quickly scale up to meet such a high demand)
• If using mySql, set up mySql connections so that they run sequentially so that slow queries won't bog down the system

What else am I missing? I would love information about specific tools and configuration options (again, using Linux here), and/or anything that is specific to Amazon EC2.

ps: Notes about monitoring for DDOS would also be welcomed - perhaps with nagios? ;)

Answer 1

A DDOS (or even a DOS), in its essence, is a resource exhaustion. You will never be able to eliminate bottlenecks, as you can only push them farther away.

On AWS, you are lucky because the network component is very strong - it be very surprising to learn that the upstream link was saturated). However, the CPU, as well as disks I/O, are way easier to flood.

The best course of action would be by starting some monitoring (local such as SAR, remote with Nagios and/or ScoutApp) and some remote logging facilities (syslog-ng). With such setup, you will be able to identify which resources get saturated (network socket due to Syn flood ; CPU due to bad SQL queries or crawlers ; ram due to …). Don’t forget to have your log partition (if you don’t have remote logging enable) on an EBS volumes (to later study the logs).

If the attack come through a web pages, the access log (or the equivalent) can be very useful.

Answer 2

You can also further isolate your EC2 instances by putting them behind an Elastic Load Balancer and only accepting traffic from the ELB instance. This puts more of the onus on Amazon to manage DDOS attacks.

I assume that you'll still have SSH open to all, so it's likely you'll still see some rogue traffic coming in there, unless you can lock down that port to some static IPs. You could change the SSHd port to something more obscure (i.e., something other than 22) to further reduce DDOS hits (most bots only check known ports).

I'll also mention fail2ban, which can monitor logs and temporarily modify your ip tables to block specific IPs (for instance, if there have been 6 failed attempts to SSH into your host from a single IP address, it can block that IP for 30 minutes or so).

Answer 3

@cwd:

If you are using Apache, I suggest mod_security. Packaged by most vendors, the core rules set do an fantastic job.

Another hardening step, is ;imiting the requests at the webserver level. Nginx., Apache can throttle and limit incoming requests.

Answer 4

I would highly recommend using Cloudflare: http://www.cloudflare.com/features-security

"CloudFlare leverages the knowledge of a diverse community of websites to power a new type of security service. Online threats range from nuisances like comment spam and excessive bot crawling to malicious attacks like SQL injection and denial of service (DOS) attacks. CloudFlare provides security protection against all of these types of threats and more to keep your website safe."