In computer networks, rate limiting is used to control the rate of traffic sent or received on a network interface.

learn more… | top users | synonyms (2)

16
votes
1answer
8k views

NGINX - throttle requests to prevent abuse

The reason I want to do this is because users develop against our API with JavaScript, and some developers screw up and cause visitors to slam the server with AJAX requests. When this happens, I want ...
9
votes
6answers
14k views

Postfix throttling for outgoing messages

I need Postfix to send outgoing messages (from local PHP) with a certain rate. Say, one message in 120 seconds. Any messages exceeding this rate should be queued (delayed) and delivered later. ...
7
votes
3answers
4k views

Rate limiting with UFW: setting limits

UFW's man page mentions that it can setup iptables rate limiting for me: ufw supports connection rate limiting, which is useful for protecting against brute-force login attacks. ufw ...
7
votes
2answers
544 views

Can I throttle tar without ionice?

My tar command tar --remove-files -cvif 2011-08-02_14-05-09.tar 2011-08-02_14-05-09 Writes 80MB/s. That's beyond the capacity we can spare because MySQL starts queuing up writes and eventually we ...
5
votes
2answers
7k views

Apache2: Limit simultaneous requests & throttle bandwidth per IP/client?

I want to limit simultaneous requests & throttle bandwidth per IP/Client on a single apache vhost. In other words, I want to ensure that this site, which hosts large media files, doesn't get ...
5
votes
1answer
607 views

Mysterious bandwidth limit

For some unknown reason, our server box running Windows Server 2008 R2 has a per connection bandwidth limit of 0.2 MB/s. Meaning, while one TCP connection can pull at max 0.2 MB/s, 60 parallel ...
4
votes
2answers
1k views

rsync ionice destination

To avoid rsync to starve the system and network we use ionice when starting rsync and set the --bwlimit parameter. For example: ionice -c2 -n7 rsync -aH --bwlimit=30000 /foo root@dest.com:/ This ...
4
votes
1answer
699 views

using trickle to slow down browser

according to trickle's man page, http://linux.die.net/man/1/trickle i can limit the download speed of a process, e.g. trickle -u 10 -d 20 ncftp to Launch ncftp(1) limiting its upload capacity to 10 ...
4
votes
1answer
3k views

How to rate-limit in nginx, but including/excluding certain IP addresses?

I'm able to use limit_req to rate-limit all requests to my server. However I'd like to remove the rate restriction for certain IP addresses (i.e. whitelist) and use a different rate restriction for ...
3
votes
2answers
2k views

nginx rate limiting with X-Forwarded-For header

I'm looking into rate-limiting using nginx's HttpLimitReqModule. However, requests are all coming from the same IP (a loadbalancer), with the real IP address in the headers. Is there a way to have ...
3
votes
2answers
5k views

Should I rate-limit packets with iptables?

I'm using iptables on Ubuntu Server. It's a web server on a VPS. I'd like to know if I should rate-limit packets. If so, what should I rate-limit? And should I do so globally or per IP address? ...
3
votes
1answer
2k views

Rate Limiting with HP ProCurve 5406 QOS

Is it possible to enable an absolute value rate limit using QOS on a HP ProCurve 5406 switch for a particular IP address on a specific port? We have a 10Mbps head end connection for multiple ...
3
votes
2answers
803 views

iptables rate-limit module problem

I am using iptables' rate-limit module to prevent DoS attack (I know it cannot stop a full scale DDoS but at least it can help with smaller attacks). In my rules I have something like: ...
3
votes
3answers
1k views

Limit number of requests to a specific set of URLs by IP address

I am working on a site, which is going to allow downloads to users, there will be around 2,000,000 files which can be downloaded. We want to discourage people from crawling and taking all of these ...
3
votes
3answers
1k views

Throttle connections to web service if load gets too high?

I have a web site that communicates via XMLRPC with an XMLRPC server web service. (The web service is written in Python using xmlrpclib.) I believe that xmlrpclib will block while it is handling one ...
3
votes
2answers
3k views

URL-based request rate limiting in Apache

There are many rate limiting and QoS tools out there, but so far I have not been able to find one that meets my specific needs, as follows: I have a web app running on Apache 2.2 and I want each of ...
3
votes
1answer
627 views

Windows 2008: throttling network bandwidth of a process

Is it possible to limit the network bandwidth of a process on Windows 2008? We need to run a long running upload but don't want to saturate the network (we have time).
3
votes
0answers
440 views

Limiting bandwidth with hashlimit (e.g. kb/s — not connections!) doesn't work, though the man page says it should

According to the iptables-extensions man page hashlimit can do bandwidth limiting: "flows exceeding 512kbyte/s" => --hashlimit-mode srcip,dstip,srcport,dstport --hashlimit-above 512kb/s ...
2
votes
2answers
923 views

DNS query limit WinServer 2008R2

Situation: I am currently seeing a huge wave of incoming DNS requests arriving in burst of around 85 queries per 2-3 seconds from one IP. Then another IP sends a new wave and so on (currently my DNS ...
2
votes
1answer
1k views

How soon does nginx's token bucket replenish when limiting at requests per minute?

We've decided that we want to experiment and limit requests per minute instead of requests per second on our sites. However, I am confused by the burst parameter in this context. I am under the ...
2
votes
1answer
416 views

Block UDP traffic by rate/string-match on Windows 2008

I am administrating a Windows 2008 server (64-bit). It is being subjected to a focused set of DOS attacks where UDP packets are sent to a specific port and contain a specific string. What I need to ...
2
votes
1answer
883 views

throttle connections to apache load balancer

I have a load balancer using apache: http://httpd.apache.org/docs/2.2/mod/mod%5Fproxy%5Fbalancer.html The problem is our bandwidth. We're trying to get more, but the ISP has to run new lines and ...
2
votes
1answer
189 views

Best way for automated ban / throttle of aggressive clients on a Linux / Apache webserver for a period of time?

I notice occasional spikes on my server load and when I check access logs I notice a pattern for some automated forum poster that tries to repeatedly access a comment/reply URL. What would be the best ...
2
votes
2answers
971 views

Can Haproxy deny a request by IP if its stick-table is full?

In my haproxy configs I'm setting a stick-table of size 5 that stores every incoming IP address (for 1 minute), and it is set as nopurge so new entries won't get stored in the table. What I'd like to ...
2
votes
1answer
78 views

Protect a public-facing PostgreSQL server against abuse

I have a Postgres database containing public information that I'd like to expose to the internet, for anyone to use. What steps can I take to prevent: Excessively expensive queries that could hog ...
2
votes
1answer
99 views

Mail server - limit connections on port 25 to 1 PC alone

I have a situation where only one PC on a network (network is behind a single public IP address) can connect to the mail server on port 25 (send out emails using Outlook, perform successful telnet ...
2
votes
1answer
413 views

Reasonable rate limiting settings with PF firewall

I have the following settings in my pf.conf to dissuade abusive clients (I get a few a day that are obviously just scraping content indiscriminately and slow everything down): pass in on $ext_if ...
2
votes
1answer
248 views

How can I prevent postfix from sending thousands of emails?

I have a Django installation that is configured to email administrators (five of us) whenever an error occurs. All of the generated emails are directed to postfix and then on to the mail server for ...
2
votes
2answers
1k views

HAProxy stops accepting connections

I've been using HAProxy to loadbalance my appservers for many months without problems. Recently some traffic spikes have lead me to setting the maxconn parameter to rate-limit connections to my ...
2
votes
1answer
1k views

Rate Limiting Outbound E-mail Message on a WHM/cPanel/Exim/CentOS 5 system. .

I have a client who is using Interspire e-mail marketer to send out his e-mail messages, and this guy sends something like 100,000 e-mails per session and does this around 3-4 times per month. He's ...
2
votes
2answers
1k views

Know a way to parse Speed Test results?

I like SpeedTest.net, but what I'd really like to do is put a graph in cacti, run a speed test every half-hour, and graph the results (maybe running it three times each time and averaging it). I ...
2
votes
2answers
676 views

How to rate limit outgoing SMTP connections for Postfix

I have a postfix instance running on a VM. The VM provider shuts us down if we exceed a threshold of SMTP connections per minute - something like 10. In a typical day we might send 1000 messages, so ...
2
votes
0answers
1k views

VLAN based rate limiting using tc

I have been trying to perform VLAN based rate limiting on an academic project I am working on. The project uses Openflow (with Beacon controller) and the network is emulated using Mininet. The network ...
1
vote
5answers
2k views

What are some good programs which can be used to rate-limit traffic on a server?

We have some applications which hog network traffic on one of our servers (Windows 7) at the expense of the other (more important) applications. Is there an easy way to limit the bandwidth usage of ...
1
vote
2answers
672 views

What is the main difference between iptables and EC2's “security groups”?

If I block traffic via a security group, I assume I don't have to pay for it. But if I block traffic via iptables installed on my server then I assume it would be me that pays for any incoming ...
1
vote
1answer
118 views

Will traffic control software cause tcp packets to be dropped

I am testing our client-server applications using NetLimiter which is a traffic control software. I found that, when client is receiving files from server and I use NetLimiter to set the download rate ...
1
vote
1answer
654 views

linux kernel 3.0.0 i7 CPU overheats and locks up

The server with i7 CPU (Intel(R) Core(TM) i7 CPU 950 @ 3.07GHz) under linux kernel 3.0.0 (3.0.0-10-server) generally works properly, but when it is put under heavy load - it overheats (as shown by ...
1
vote
3answers
2k views

Throttling network speed when copying files to SMB mounted NAS drive

I have a fairly modest Ubuntu box (Jaunty) running as a small webserver. Each night i have a cron job which tar/gzip's important directories and does a simple cp to copy them over to a backup NAS ...
1
vote
1answer
1k views

Nginx - What is does the nodelay option do when limiting requests?

With the nginx HttpLimitReq module requests can be limited by IP. However, I'm not understanding what the "nodelay" option does. If the excess requests within the limit burst delay are not ...
1
vote
1answer
60 views

lighttpd rate limits per directory

I'm creating a site with a large number of small tools. One is a tool which uses a Python cgi-script that interacts with the Natural Language Toolkit (nltk), which is disk and CPU intensive. I have ...
1
vote
2answers
1k views

Search Engine Bot - Large amount of hits

I've started tracking user-agent strings on a website at the start of each session. Looking at the data for this month so far I'm seeing on search engine bot that keeps coming up a lot.. Mozilla/5.0 ...
1
vote
1answer
400 views

Throttle or Ignore requests based on IP

Is there something that I can use to set rules for Apache to throttle temporarily or ignore any IP address that makes X number of requests per second? It would be great if there is an Apache module ...
1
vote
2answers
720 views

Apache bandwidth throttling per client, by subnet

We're interested in restricting the number of requests per second and/or available bandwidth to HTTP clients, to stop accidental DoS. We provide free scientific data and web services, and sadly some ...
1
vote
1answer
166 views

nginx 301'ing limit_req rate limited requests instead of error page

I have a bulk search form on apache servers behind nginx. I'd like to protect that location so that the second request in a 10 minute period gets 301'd to a "Would you like to use our API" page ...
1
vote
1answer
1k views

How to rate-limit outgoing email on Linux/Postfix host?

Background: I'm a fan of David Allen's Getting Things Done, and that means when an email arrives in my inbox, if I can reply right away, I want to do so and keep my inbox clear. But there's a ...
1
vote
2answers
357 views

Is it possible to put resource limits on an IIS7 site?

I have a number of different web apps running under IIS7. Is it possible to put limits on one app - for example, limit the number of connections, bandwidth, memory usage and CPU allowed? I need to ...
1
vote
0answers
49 views

Linux Traffic Controller (Linux tc): hierarchical token bucket (htb) packet queue size

In my Linux router, I am using the following configuration to limit the rate of traffic towards port 44444 of a client in the LAN (client address is 192.168.10.2, connected through the router's eth1 ...
1
vote
0answers
220 views

Logstash, Kibana and email alerts

I am trying to setup email alerts using logstash. Right now it emails me EVERY time the pattern "Error" is parsed into my log file which can lead to a lot of unnecessary emails. I'd like to create a ...
1
vote
1answer
59 views

Relax ssh rate-limit rules for certain ip addresses

We've written some tools to automate common development tasks. One tool, for example, ensures all our git repositories are up-to-date. For this to work, our tool needs to connect to our git server and ...
1
vote
0answers
73 views

Debian exim4 rate limits

I try to limit on exim4 the outgoing mails per day per domain, i found this: acl_check_ratelimit: accept authenticated = * endpass ratelimit = 40 / 1d / per_rcpt / ...