Meta Server Fault
to customize your list.
more stack exchange communities
Start here for a quick overview of the site
Detailed answers to any questions you might have
Discuss the workings and policies of this site
What to do about spoofed user agents? Scrapers pretending to be spiders
I've been following a few spiders in our logs and I did a traceroute on their ip to find out they are in fact EC2 instances. The user agents are listed as Google bot and msnbot but they are not ...
Mar 21 '11 at 21:15
newest spider scraping questions feed
Welcome Shane Madden, ServerFault Valued Associate #0000007!
Putting the Community back in Wiki
Hot Network Questions
fractions data type -- when to reduce?
Could human activity cause earth's core lose its heat?
Can I prevent diagonal movement from exploring more of the map?
Is there a single word that means “hitting the target but missing the point”?
cd .. on root folder
Realised afterwards that I committed mild plagiarism in a paper I wrote as an undergraduate, what should I do?
What time counts when it comes to remote work?
When you discard cards in the beginning of the game, where do they go?
Search vs. Look Up
How to measure the curvature of the space-time?
Should I obscure database primary keys (IDs) in application front end?
Is the determinant differentiable?
Python3 rounding to nearest even
Can I set up conditional forwarding for a parent DNS zone?
What French phrase is the origin of "gardyloo?"
Is there a computable ordinal encoding the proof strength of ZF? Is it knowable?
Perplexing Chimera + Hypnotic Siren. Do I control all of my opponent's spells?
Can we call someone X太太 or not?
Sending I2C reliabily over Cat5 cables
What skill would be used when looking for an item that is among many items?
VMware ESXi SSD on RAID showing as non-ssd
Does centrifugal force exist?
Do banks give us interest even for the money that we only had briefly in our account?
Handling plagiarism as a TA
more hot questions
Life / Arts
Culture / Recreation
TeX - LaTeX
Unix & Linux
Ask Different (Apple)
Geographic Information Systems
Science Fiction & Fantasy
Seasoned Advice (cooking)
Personal Finance & Money
English Language & Usage
Mi Yodeya (Judaism)
Cross Validated (stats)
Theoretical Computer Science
Meta Stack Exchange
Stack Overflow Careers
site design / logo © 2014 stack exchange inc; user contributions licensed under
cc by-sa 3.0