0

I have a few files that can be downloaded from my website. Every few hours all files are downloaded from the same IP address (98.137.207.97 - might be Yahoo?). How should I proceed? Should I just block the IP (and hope the downloads are not taken up by another IP) or are there better ways to handle this?

1

You should create a robot.txt file if you don't want bots/spiders/etc to be crawling certain parts of your site.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.