In computer networks, rate limiting is used to control the rate of traffic sent or received on a network interface.

learn more… | top users | synonyms (1)

22
votes
1answer
15k views

NGINX - throttle requests to prevent abuse

The reason I want to do this is because users develop against our API with JavaScript, and some developers screw up and cause visitors to slam the server with AJAX requests. When this happens, I want ...
15
votes
3answers
9k views

Rate limiting with UFW: setting limits

UFW's man page mentions that it can setup iptables rate limiting for me: ufw supports connection rate limiting, which is useful for protecting against brute-force login attacks. ufw ...
12
votes
2answers
7k views

How to rate-limit in nginx, but including/excluding certain IP addresses?

I'm able to use limit_req to rate-limit all requests to my server. However I'd like to remove the rate restriction for certain IP addresses (i.e. whitelist) and use a different rate restriction for ...
9
votes
2answers
5k views

nginx rate limiting with X-Forwarded-For header

I'm looking into rate-limiting using nginx's HttpLimitReqModule. However, requests are all coming from the same IP (a loadbalancer), with the real IP address in the headers. Is there a way to have ...
9
votes
6answers
18k views

Postfix throttling for outgoing messages

I need Postfix to send outgoing messages (from local PHP) with a certain rate. Say, one message in 120 seconds. Any messages exceeding this rate should be queued (delayed) and delivered later. ...
7
votes
2answers
651 views

Can I throttle tar without ionice?

My tar command tar --remove-files -cvif 2011-08-02_14-05-09.tar 2011-08-02_14-05-09 Writes 80MB/s. That's beyond the capacity we can spare because MySQL starts queuing up writes and eventually we ...
6
votes
2answers
3k views

rsync ionice destination

To avoid rsync to starve the system and network we use ionice when starting rsync and set the --bwlimit parameter. For example: ionice -c2 -n7 rsync -aH --bwlimit=30000 /foo root@dest.com:/ This ...
6
votes
3answers
7k views

Should I rate-limit packets with iptables?

I'm using iptables on Ubuntu Server. It's a web server on a VPS. I'd like to know if I should rate-limit packets. If so, what should I rate-limit? And should I do so globally or per IP address? ...
5
votes
2answers
7k views

How to rate limit outgoing SMTP connections for Postfix

I have a postfix instance running on a VM. The VM provider shuts us down if we exceed a threshold of SMTP connections per minute - something like 10. In a typical day we might send 1000 messages, so ...
5
votes
2answers
9k views

Apache2: Limit simultaneous requests & throttle bandwidth per IP/client?

I want to limit simultaneous requests & throttle bandwidth per IP/Client on a single apache vhost. In other words, I want to ensure that this site, which hosts large media files, doesn't get ...
5
votes
2answers
2k views

using trickle to slow down browser

according to trickle's man page, http://linux.die.net/man/1/trickle i can limit the download speed of a process, e.g. trickle -u 10 -d 20 ncftp to Launch ncftp(1) limiting its upload capacity to 10 ...
5
votes
1answer
1k views

Mysterious bandwidth limit

For some unknown reason, our server box running Windows Server 2008 R2 has a per connection bandwidth limit of 0.2 MB/s. Meaning, while one TCP connection can pull at max 0.2 MB/s, 60 parallel ...
4
votes
3answers
3k views

Nginx - What is does the nodelay option do when limiting requests?

With the nginx HttpLimitReq module requests can be limited by IP. However, I'm not understanding what the "nodelay" option does. If the excess requests within the limit burst delay are not ...
4
votes
2answers
2k views

How soon does nginx's token bucket replenish when limiting at requests per minute?

We've decided that we want to experiment and limit requests per minute instead of requests per second on our sites. However, I am confused by the burst parameter in this context. I am under the ...
4
votes
1answer
484 views

Common back pressure strategies for services under sustained high load

I'm wondering what are the common backpressure strategies people use for their web services? Imagine your service operates under heavy load, and at some point the load reaches 120% of your capacity. ...
4
votes
1answer
5k views

Nginx $http_x_forwarded_for not always being set

I've noticed in our access logs that sometimes, the $http_x_forwarded_for variable has not been set, and instead, the correct IP is the one set to $remote_addr. Is it possible to set ...
4
votes
1answer
256 views

nginx and cloudlfare ip confusion?

I am using the below config in http block of my nginx.conf to get real ip while using Cloudflare acceleration service: ## Cloudflare settings to get Real IP's set_real_ip_from ...
3
votes
1answer
157 views

What rate limiting options are provided by ISC BIND?

This is a Canonical Question about rate limiting in the ISC BIND nameserver software. Recently I've heard of BIND's rate limiting features. DNS RRL in particular seems to be becoming more ...
3
votes
2answers
2k views

Is there a way to rate limit connection attempts with firewalld?

On our CentOS 6 servers, I've used information from this article to reduce the brute force ssh attempts on our servers, specifically the rate limiting / logging section. Is there a way to do the same ...
3
votes
1answer
13k views

Logstash, Kibana and email alerts

I am trying to setup email alerts using logstash. Right now it emails me EVERY time the pattern "Error" is parsed into my log file which can lead to a lot of unnecessary emails. I'd like to create a ...
3
votes
3answers
2k views

Limit number of requests to a specific set of URLs by IP address

I am working on a site, which is going to allow downloads to users, there will be around 2,000,000 files which can be downloaded. We want to discourage people from crawling and taking all of these ...
3
votes
2answers
3k views

Rate limit exim per user basis

How can I implement "rate limit" in Exim so that every user in my exim can send specific number of emails per day. List A of Users can send 100 emails per day and the list B of users can send 500 ...
3
votes
1answer
3k views

Rate Limiting with HP ProCurve 5406 QOS

Is it possible to enable an absolute value rate limit using QOS on a HP ProCurve 5406 switch for a particular IP address on a specific port? We have a 10Mbps head end connection for multiple ...
3
votes
2answers
1k views

iptables rate-limit module problem

I am using iptables' rate-limit module to prevent DoS attack (I know it cannot stop a full scale DDoS but at least it can help with smaller attacks). In my rules I have something like: ...
3
votes
3answers
1k views

Throttle connections to web service if load gets too high?

I have a web site that communicates via XMLRPC with an XMLRPC server web service. (The web service is written in Python using xmlrpclib.) I believe that xmlrpclib will block while it is handling one ...
3
votes
1answer
770 views

nginx 301'ing limit_req rate limited requests instead of error page

I have a bulk search form on apache servers behind nginx. I'd like to protect that location so that the second request in a 10 minute period gets 301'd to a "Would you like to use our API" page ...
3
votes
3answers
266 views

Block ANY request in Bind

I'm having trouble figuring out how to block ANY dns request on BIND to lower the changes of a DNS Amplification attacks. I've seen guides online but most of them appear to perform rate limit via ...
3
votes
2answers
4k views

URL-based request rate limiting in Apache

There are many rate limiting and QoS tools out there, but so far I have not been able to find one that meets my specific needs, as follows: I have a web app running on Apache 2.2 and I want each of ...
3
votes
2answers
3k views

HAProxy stops accepting connections

I've been using HAProxy to loadbalance my appservers for many months without problems. Recently some traffic spikes have lead me to setting the maxconn parameter to rate-limit connections to my ...
3
votes
1answer
889 views

Windows 2008: throttling network bandwidth of a process

Is it possible to limit the network bandwidth of a process on Windows 2008? We need to run a long running upload but don't want to saturate the network (we have time).
3
votes
1answer
941 views

Limiting bandwidth with hashlimit (e.g. kb/s — not connections!) doesn't work, though the man page says it should

According to the iptables-extensions man page hashlimit can do bandwidth limiting: "flows exceeding 512kbyte/s" => --hashlimit-mode srcip,dstip,srcport,dstport --hashlimit-above 512kb/s ...
3
votes
1answer
2k views

VLAN based rate limiting using tc

I have been trying to perform VLAN based rate limiting on an academic project I am working on. The project uses Openflow (with Beacon controller) and the network is emulated using Mininet. The network ...
2
votes
2answers
1k views

What is the main difference between iptables and EC2's “security groups”?

If I block traffic via a security group, I assume I don't have to pay for it. But if I block traffic via iptables installed on my server then I assume it would be me that pays for any incoming ...
2
votes
1answer
1k views

Is there a way to rate limit connections with HAProxy using multiple thresholds

I've implemented simple rate limiting using HaProxy in a similar fashion to the way StackExchange does it with HaProxy. I'm trying to make it a bit more advanced so that there are multiple thresholds ...
2
votes
2answers
2k views

DNS query limit WinServer 2008R2

Situation: I am currently seeing a huge wave of incoming DNS requests arriving in burst of around 85 queries per 2-3 seconds from one IP. Then another IP sends a new wave and so on (currently my DNS ...
2
votes
1answer
661 views

Block UDP traffic by rate/string-match on Windows 2008

I am administrating a Windows 2008 server (64-bit). It is being subjected to a focused set of DOS attacks where UDP packets are sent to a specific port and contain a specific string. What I need to ...
2
votes
1answer
1k views

throttle connections to apache load balancer

I have a load balancer using apache: http://httpd.apache.org/docs/2.2/mod/mod%5Fproxy%5Fbalancer.html The problem is our bandwidth. We're trying to get more, but the ISP has to run new lines and ...
2
votes
1answer
338 views

Configuring RRL on a DNS server using BIND

I'd like to configure a DNS server to use Response Rate Limiting (RRL) to mitigate DNS DDoS attacks. Whatever, I have to configure it top use RRL with the 'slip' parameter at 2. I've made some ...
2
votes
1answer
256 views

Best way for automated ban / throttle of aggressive clients on a Linux / Apache webserver for a period of time?

I notice occasional spikes on my server load and when I check access logs I notice a pattern for some automated forum poster that tries to repeatedly access a comment/reply URL. What would be the best ...
2
votes
2answers
1k views

Can Haproxy deny a request by IP if its stick-table is full?

In my haproxy configs I'm setting a stick-table of size 5 that stores every incoming IP address (for 1 minute), and it is set as nopurge so new entries won't get stored in the table. What I'd like to ...
2
votes
1answer
119 views

Rate limiting using haproxy per url per ip

Our servers are being hit by someone constantly. Its not DDOS. Just one ip hits one url more than 200 times a second. Currently I am blocking the user by ip tables. We are using HAproxy for load ...
2
votes
1answer
356 views

Postfix - rate limit outgoing concurrent connections to one machine/ip

I need to limit postfix outgoing connections for specific mailservers, but it seems to me that posfix's limiting is based on domains? What if there is one mailserver for more domains? I tried to find ...
2
votes
1answer
88 views

Postfix and smtpd_client_rate_limit to detect compromissed accounts

Some of our clients are administrators who need to send corporative messages to no more than 300 clients, specifying all of these clients together in the "To:" fields, instead of individual messages ...
2
votes
1answer
219 views

Protect a public-facing PostgreSQL server against abuse

I have a Postgres database containing public information that I'd like to expose to the internet, for anyone to use. What steps can I take to prevent: Excessively expensive queries that could hog ...
2
votes
1answer
129 views

Mail server - limit connections on port 25 to 1 PC alone

I have a situation where only one PC on a network (network is behind a single public IP address) can connect to the mail server on port 25 (send out emails using Outlook, perform successful telnet ...
2
votes
1answer
763 views

Reasonable rate limiting settings with PF firewall

I have the following settings in my pf.conf to dissuade abusive clients (I get a few a day that are obviously just scraping content indiscriminately and slow everything down): pass in on $ext_if ...
2
votes
1answer
362 views

How can I prevent postfix from sending thousands of emails?

I have a Django installation that is configured to email administrators (five of us) whenever an error occurs. All of the generated emails are directed to postfix and then on to the mail server for ...
2
votes
1answer
2k views

Rate Limiting Outbound E-mail Message on a WHM/cPanel/Exim/CentOS 5 system. .

I have a client who is using Interspire e-mail marketer to send out his e-mail messages, and this guy sends something like 100,000 e-mails per session and does this around 3-4 times per month. He's ...
2
votes
1answer
959 views

How can I use fail2ban to block scrapers?

I have a media site and problems of users coming along and scraping all of the content.I placed a invisible URL on the page to catch spiders that immediately blocks the ip, but some people have ...
2
votes
2answers
1k views

Know a way to parse Speed Test results?

I like SpeedTest.net, but what I'd really like to do is put a graph in cacti, run a speed test every half-hour, and graph the results (maybe running it three times each time and averaging it). I ...