Many large scale sites block access if too many requests are made from one origin. But large provider proxies would exhibit similiar access. So how should a server differ between those requests? And what behaveour is needed for legitimate proxy need to not trigger blocking?
Most sites check for open proxies, maybe crawl some proxy lists and check various blacklists.
Then as time goes and abuses go the admins just blacklist and/or throttle IP blocks. For example if you get a lot of shady traffic from China and your site is not even targeted at them then it's a fairly common practice to block traffic from their IP ranges (IANA and here's a list that resulted from someone digging through APNIC records).
After this it boils down to what site do you run and how important those users might be for your business. Google just uses CAPTCHAs to limit resource abuse, and I'm sure they'd utilize more drastic measures (even to the point of altering AS PATHs via BGP) if must.
So if you can authenticate the users behind the IP and their behavior is consistent with normal usage, then it's a probably just a transparent enterprise/corporate proxy for some office. (For monitoring usage patterns you might try using some kind of IPS/IDS with custom HTTP filters tailored to your needs.)
They would probably add the IP Addresses of the proxies to a whitelist of some sort so they are ignored by the traffic filtering software.
A more general question is "How do I deal with with receiving both malicious and benign traffic from the same IP address(es)?"
Besides open (and paid) proxies, you can have a legitimate user who unknowingly also hosts a spam bot. Or a company with a large NAT hiding spam bots. Or an entire country that sends you both spam and real users.
These can also come in different ratios. Some examples from the site I run: So far, 100% of all traffic I have received from WebSense IP address has been spammers and apart from the one customer I have from Senegal, the rest have been spammers. On the other hand China sends me about 25% real traffic and 75% spam bots.
Blocking all requests from these IPs or netblocks will always have a false positive rate, however you don't have to block all requests when you block an IP address. If you are having a problem with blog comment or forum spam, you could try limiting
Rather than blocking the IP address(es) you could use the IP address as part of your spam scoring system. If you have a Bayesian scoring system for spam, just add the user's IP address into the tokens. Addresses that always spam will get a high score, addresses that don't will get a low score and addresses that have both types of users (such as proxies) will hover around the middle so that they are judged on the content of their messages and not their IP address.
You could even ignore IP addresses entirely and just judge all submitted content on the content itself and not the reputation of the IP address it came from.
Although I haven't done it myself, you could probably configure fail2ban to read your spam scoring logs and drop the IP addresses of the worst offenders into a