I would like to rate-limit (but not block) various crawlers. Based on other examples of similar questions, I've come up with this:
http {
## 10MB zone, 5 reqs per sec
limit_req_zone $limit zone=zone_a:10m rate=5r/s;
geo $limited {
default 0;
157.55.39.31/32 1;
157.55.39.39/32 1;
157.55.39.40/32 1;
157.55.39.166/32 1;
157.55.39.167/32 1;
157.55.39.212/32 1;
157.55.39.213/32 1;
68.180.229.48/32 1;
}
map $limited $limit {
0 "false";
1 "true";
}
## - do something -
}
If I understand it correctly, a matched IP in the geo-block will set $limited to 1 and then the map-block will set $limit to true/false. I think what needs to happen next is an 'if' statement like this:
if ($limit = "true") {
limit_rate zone=zone_a burst=10;
}
Is that the correct way to do this? Is there a downside to rate-limiting?