0

I have a few files that can be downloaded from my website. Every few hours all files are downloaded from the same IP address (98.137.207.97 - might be Yahoo?). How should I proceed? Should I just block the IP (and hope the downloads are not taken up by another IP) or are there better ways to handle this?

1 Answer 1

1

You should create a robot.txt file if you don't want bots/spiders/etc to be crawling certain parts of your site.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .