In computer networks, rate limiting is used to control the rate of traffic sent or received on a network interface.

learn more… | top users | synonyms (1)

18
votes
1answer
11k views

NGINX - throttle requests to prevent abuse

The reason I want to do this is because users develop against our API with JavaScript, and some developers screw up and cause visitors to slam the server with AJAX requests. When this happens, I want ...
9
votes
3answers
6k views

Rate limiting with UFW: setting limits

UFW's man page mentions that it can setup iptables rate limiting for me: ufw supports connection rate limiting, which is useful for protecting against brute-force login attacks. ufw ...
9
votes
6answers
15k views

Postfix throttling for outgoing messages

I need Postfix to send outgoing messages (from local PHP) with a certain rate. Say, one message in 120 seconds. Any messages exceeding this rate should be queued (delayed) and delivered later. ...
7
votes
2answers
601 views

Can I throttle tar without ionice?

My tar command tar --remove-files -cvif 2011-08-02_14-05-09.tar 2011-08-02_14-05-09 Writes 80MB/s. That's beyond the capacity we can spare because MySQL starts queuing up writes and eventually we ...
7
votes
2answers
4k views

How to rate-limit in nginx, but including/excluding certain IP addresses?

I'm able to use limit_req to rate-limit all requests to my server. However I'd like to remove the rate restriction for certain IP addresses (i.e. whitelist) and use a different rate restriction for ...
5
votes
2answers
8k views

Apache2: Limit simultaneous requests & throttle bandwidth per IP/client?

I want to limit simultaneous requests & throttle bandwidth per IP/Client on a single apache vhost. In other words, I want to ensure that this site, which hosts large media files, doesn't get ...
5
votes
2answers
2k views

rsync ionice destination

To avoid rsync to starve the system and network we use ionice when starting rsync and set the --bwlimit parameter. For example: ionice -c2 -n7 rsync -aH --bwlimit=30000 /foo root@dest.com:/ This ...
5
votes
2answers
1k views

using trickle to slow down browser

according to trickle's man page, http://linux.die.net/man/1/trickle i can limit the download speed of a process, e.g. trickle -u 10 -d 20 ncftp to Launch ncftp(1) limiting its upload capacity to 10 ...
5
votes
1answer
893 views

Mysterious bandwidth limit

For some unknown reason, our server box running Windows Server 2008 R2 has a per connection bandwidth limit of 0.2 MB/s. Meaning, while one TCP connection can pull at max 0.2 MB/s, 60 parallel ...
4
votes
2answers
3k views

nginx rate limiting with X-Forwarded-For header

I'm looking into rate-limiting using nginx's HttpLimitReqModule. However, requests are all coming from the same IP (a loadbalancer), with the real IP address in the headers. Is there a way to have ...
4
votes
2answers
6k views

Should I rate-limit packets with iptables?

I'm using iptables on Ubuntu Server. It's a web server on a VPS. I'd like to know if I should rate-limit packets. If so, what should I rate-limit? And should I do so globally or per IP address? ...
4
votes
1answer
2k views

Nginx $http_x_forwarded_for not always being set

I've noticed in our access logs that sometimes, the $http_x_forwarded_for variable has not been set, and instead, the correct IP is the one set to $remote_addr. Is it possible to set ...
3
votes
2answers
2k views

Nginx - What is does the nodelay option do when limiting requests?

With the nginx HttpLimitReq module requests can be limited by IP. However, I'm not understanding what the "nodelay" option does. If the excess requests within the limit burst delay are not ...
3
votes
1answer
2k views

Rate Limiting with HP ProCurve 5406 QOS

Is it possible to enable an absolute value rate limit using QOS on a HP ProCurve 5406 switch for a particular IP address on a specific port? We have a 10Mbps head end connection for multiple ...
3
votes
2answers
973 views

iptables rate-limit module problem

I am using iptables' rate-limit module to prevent DoS attack (I know it cannot stop a full scale DDoS but at least it can help with smaller attacks). In my rules I have something like: ...
3
votes
1answer
2k views

How soon does nginx's token bucket replenish when limiting at requests per minute?

We've decided that we want to experiment and limit requests per minute instead of requests per second on our sites. However, I am confused by the burst parameter in this context. I am under the ...
3
votes
3answers
1k views

Throttle connections to web service if load gets too high?

I have a web site that communicates via XMLRPC with an XMLRPC server web service. (The web service is written in Python using xmlrpclib.) I believe that xmlrpclib will block while it is handling one ...
3
votes
2answers
137 views

Is there a way to rate limit connection attempts with firewalld?

On our CentOS 6 servers, I've used information from this article to reduce the brute force ssh attempts on our servers, specifically the rate limiting / logging section. Is there a way to do the same ...
3
votes
3answers
1k views

Limit number of requests to a specific set of URLs by IP address

I am working on a site, which is going to allow downloads to users, there will be around 2,000,000 files which can be downloaded. We want to discourage people from crawling and taking all of these ...
3
votes
2answers
4k views

How to rate limit outgoing SMTP connections for Postfix

I have a postfix instance running on a VM. The VM provider shuts us down if we exceed a threshold of SMTP connections per minute - something like 10. In a typical day we might send 1000 messages, so ...
3
votes
2answers
2k views

HAProxy stops accepting connections

I've been using HAProxy to loadbalance my appservers for many months without problems. Recently some traffic spikes have lead me to setting the maxconn parameter to rate-limit connections to my ...
3
votes
2answers
3k views

URL-based request rate limiting in Apache

There are many rate limiting and QoS tools out there, but so far I have not been able to find one that meets my specific needs, as follows: I have a web app running on Apache 2.2 and I want each of ...
3
votes
1answer
756 views

Windows 2008: throttling network bandwidth of a process

Is it possible to limit the network bandwidth of a process on Windows 2008? We need to run a long running upload but don't want to saturate the network (we have time).
3
votes
1answer
162 views

Common back pressure strategies for services under sustained high load

I'm wondering what are the common backpressure strategies people use for their web services? Imagine your service operates under heavy load, and at some point the load reaches 120% of your capacity. ...
3
votes
1answer
181 views

nginx and cloudlfare ip confusion?

I am using the below config in http block of my nginx.conf to get real ip while using Cloudflare acceleration service: ## Cloudflare settings to get Real IP's set_real_ip_from ...
3
votes
1answer
651 views

Limiting bandwidth with hashlimit (e.g. kb/s — not connections!) doesn't work, though the man page says it should

According to the iptables-extensions man page hashlimit can do bandwidth limiting: "flows exceeding 512kbyte/s" => --hashlimit-mode srcip,dstip,srcport,dstport --hashlimit-above 512kb/s ...
2
votes
2answers
1k views

DNS query limit WinServer 2008R2

Situation: I am currently seeing a huge wave of incoming DNS requests arriving in burst of around 85 queries per 2-3 seconds from one IP. Then another IP sends a new wave and so on (currently my DNS ...
2
votes
1answer
6k views

Logstash, Kibana and email alerts

I am trying to setup email alerts using logstash. Right now it emails me EVERY time the pattern "Error" is parsed into my log file which can lead to a lot of unnecessary emails. I'd like to create a ...
2
votes
1answer
530 views

Block UDP traffic by rate/string-match on Windows 2008

I am administrating a Windows 2008 server (64-bit). It is being subjected to a focused set of DOS attacks where UDP packets are sent to a specific port and contain a specific string. What I need to ...
2
votes
1answer
931 views

throttle connections to apache load balancer

I have a load balancer using apache: http://httpd.apache.org/docs/2.2/mod/mod%5Fproxy%5Fbalancer.html The problem is our bandwidth. We're trying to get more, but the ISP has to run new lines and ...
2
votes
1answer
85 views

Configuring RRL on a DNS server using BIND

I'd like to configure a DNS server to use Response Rate Limiting (RRL) to mitigate DNS DDoS attacks. Whatever, I have to configure it top use RRL with the 'slip' parameter at 2. I've made some ...
2
votes
1answer
229 views

Best way for automated ban / throttle of aggressive clients on a Linux / Apache webserver for a period of time?

I notice occasional spikes on my server load and when I check access logs I notice a pattern for some automated forum poster that tries to repeatedly access a comment/reply URL. What would be the best ...
2
votes
2answers
1k views

Can Haproxy deny a request by IP if its stick-table is full?

In my haproxy configs I'm setting a stick-table of size 5 that stores every incoming IP address (for 1 minute), and it is set as nopurge so new entries won't get stored in the table. What I'd like to ...
2
votes
1answer
54 views

Postfix and smtpd_client_rate_limit to detect compromissed accounts

Some of our clients are administrators who need to send corporative messages to no more than 300 clients, specifying all of these clients together in the "To:" fields, instead of individual messages ...
2
votes
1answer
140 views

Protect a public-facing PostgreSQL server against abuse

I have a Postgres database containing public information that I'd like to expose to the internet, for anyone to use. What steps can I take to prevent: Excessively expensive queries that could hog ...
2
votes
1answer
110 views

Mail server - limit connections on port 25 to 1 PC alone

I have a situation where only one PC on a network (network is behind a single public IP address) can connect to the mail server on port 25 (send out emails using Outlook, perform successful telnet ...
2
votes
1answer
592 views

Reasonable rate limiting settings with PF firewall

I have the following settings in my pf.conf to dissuade abusive clients (I get a few a day that are obviously just scraping content indiscriminately and slow everything down): pass in on $ext_if ...
2
votes
1answer
302 views

How can I prevent postfix from sending thousands of emails?

I have a Django installation that is configured to email administrators (five of us) whenever an error occurs. All of the generated emails are directed to postfix and then on to the mail server for ...
2
votes
1answer
1k views

Rate Limiting Outbound E-mail Message on a WHM/cPanel/Exim/CentOS 5 system. .

I have a client who is using Interspire e-mail marketer to send out his e-mail messages, and this guy sends something like 100,000 e-mails per session and does this around 3-4 times per month. He's ...
2
votes
1answer
827 views

How can I use fail2ban to block scrapers?

I have a media site and problems of users coming along and scraping all of the content.I placed a invisible URL on the page to catch spiders that immediately blocks the ip, but some people have ...
2
votes
2answers
1k views

Know a way to parse Speed Test results?

I like SpeedTest.net, but what I'd really like to do is put a graph in cacti, run a speed test every half-hour, and graph the results (maybe running it three times each time and averaging it). I ...
2
votes
2answers
245 views

Easy Apache module to throttle requests based on machine load?

I have an Apache server that is serving a django application with mod_wsgi. I'm wondering if there is a simple way to throttle requests at the apache level based on the machine's load. Ideally I want ...
2
votes
0answers
1k views

VLAN based rate limiting using tc

I have been trying to perform VLAN based rate limiting on an academic project I am working on. The project uses Openflow (with Beacon controller) and the network is emulated using Mininet. The network ...
1
vote
5answers
2k views

What are some good programs which can be used to rate-limit traffic on a server?

We have some applications which hog network traffic on one of our servers (Windows 7) at the expense of the other (more important) applications. Is there an easy way to limit the bandwidth usage of ...
1
vote
2answers
886 views

What is the main difference between iptables and EC2's “security groups”?

If I block traffic via a security group, I assume I don't have to pay for it. But if I block traffic via iptables installed on my server then I assume it would be me that pays for any incoming ...
1
vote
1answer
175 views

Will traffic control software cause tcp packets to be dropped

I am testing our client-server applications using NetLimiter which is a traffic control software. I found that, when client is receiving files from server and I use NetLimiter to set the download rate ...
1
vote
1answer
701 views

linux kernel 3.0.0 i7 CPU overheats and locks up

The server with i7 CPU (Intel(R) Core(TM) i7 CPU 950 @ 3.07GHz) under linux kernel 3.0.0 (3.0.0-10-server) generally works properly, but when it is put under heavy load - it overheats (as shown by ...
1
vote
3answers
2k views

Throttling network speed when copying files to SMB mounted NAS drive

I have a fairly modest Ubuntu box (Jaunty) running as a small webserver. Each night i have a cron job which tar/gzip's important directories and does a simple cp to copy them over to a backup NAS ...
1
vote
1answer
83 views

lighttpd rate limits per directory

I'm creating a site with a large number of small tools. One is a tool which uses a Python cgi-script that interacts with the Natural Language Toolkit (nltk), which is disk and CPU intensive. I have ...
1
vote
2answers
1k views

Search Engine Bot - Large amount of hits

I've started tracking user-agent strings on a website at the start of each session. Looking at the data for this month so far I'm seeing on search engine bot that keeps coming up a lot.. Mozilla/5.0 ...