I have an AWS server that is currently under DDoS via DNS amplification. I've setup CloudWatch logs for the VPC ACL and it's logging an enormous amount of rejected DNS traffic. Despite that traffic being rejected, my primary server is unreachable.

I have a secondary server on the same VPC and subnet that can be reached without any problem.

Why is it that I can access one but not the other? The ACL should be filtering the traffic at the subnet level. So if one is unreachable then they both should be unreachable, but that's not the case.

And how does one mitigate a DNS amplification attack on AWS? AWS certainly has big enough pipes. Why is the ACL not doing the job?

  • 5
    What did AWS support say? – EEAA Dec 20 '15 at 12:30
  • Is DNS amplification the only attack vector? – Jukka Dec 20 '15 at 19:39
  • Haven't talked to AWS support. I suppose I should do that next. Just wondering if there's something I don't understand about the VPC ACL. DNS amplification is the only vector that is working. I've been able to mitigate everything else so far. – Brent Dec 20 '15 at 22:06

I would add a caching/DDOS protection layer i.e. Cloudflare to the mix. They have stop most DDOS attacks to my sites in the past.

  • 1
    This does not answer the question. While it may be a good idea, it does nothing to help the OP with the issue that they are experiencing right now. – EEAA Dec 20 '15 at 14:50
  • I'm familiar with CloudFlare and it's an excellent service, but my site uses websockets and the only CloudFlare plan that enables websockets is the top of the line enterprise plan which is not very cost effective for me. – Brent Dec 20 '15 at 22:03

I ended up solving the issue.

There were a couple issues actually. I had only blocked UDP port 53 (DNS) and as it turns out there were other ports being attacked. Since my server is just a web server I was able to block all UDP traffic in the ACL. That solved one side of the attack.

They were also overloading my web server with large post requests from compromised WordPress installs. I was able to add a few lines to my Nginx configuration that dropped requests from WordPress user agents and also block large post requests.

These were the settings I used in the http section of the Nginx config

    client_max_body_size 10k;
    client_body_buffer_size 10k;
    client_header_buffer_size 1k;
    large_client_header_buffers 2 1k;
    client_body_timeout 6;
    client_header_timeout 6;
    keepalive_timeout 5;
    send_timeout 10;

Then in the server section of the Nginx config I setup a drop for WordPress and wget initiated requests

if ($http_user_agent ~* (wordpress|wget)) {
   return 403;

These settings made the server a lot more difficult to bring down

I also used iptables to rate limit incoming http connections

# Rate limit new connections to port 80
-A INPUT -p tcp -m recent --dport 80 -m state --state NEW --set
-A INPUT -p tcp -m recent --dport 80 -m state --state NEW --update --seconds 20 --hitcount 20 -j DROP

Then I used iptables to limit the maximum number of simultaneous connections to port 80.

# Limit concurrent connections for a class B to port 80
-A INPUT -p tcp -m tcp --dport 80 --tcp-flags FIN,SYN,RST,ACK SYN -m connlimit --connlimit-above 10 --connlimit-mask 16 --connlimit-saddr -j REJECT --reject-with tcp-reset
-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT

These things together made my servers much more difficult to DDoS. I'm now using multiple front end servers to reverse proxy requests to a backend server. I setup DNS round robin to expose the multiple IP addresses. This last additional step increased the amount of total bandwidth I could handle in an attack that got past all the other defenses.

So far the remaining attacks have not been able to take down my server.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.