We are experiencing a problem with scraping on our e-commerce platform.
It's not the scraping that is the problem per se, it's that whoever is doing it is spoofing a stale referrer full of our custom analytics parameters. This is playing havoc with our in-house analytics package.
All of the IPs are originating from 1e100.net
, i.e., Google IP addresses.
I want to block all these addresses from I'm concerned that it's possible I may inadvertently stop Google company resources from accessing our site. These might include, for example, resources that index our site, or other resources owned by the Adwords/Analytics team.
Is there any way to differentiate between Google App Engine traffic (which is what I suspect this traffic is), and Google company traffic.