Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I've got a web based application that I've developed and I'm seeing a lot of wget requests on the site. Should I be worried that someone is trying to hack/rip the site. I don't know what they would do with it, the "intelligence" of the system in all on the server side.

But it makes me nervous. Should I be?

share|improve this question
up vote 2 down vote accepted

Wget is a standard UNIX/Linux tool used to mirror a site (can be used as web spider/web crawler), seeing lots of requests would be consistent with such use. You should worry only when most of those HTTP GETs are for pages that don't exist or come for typical web administration packets (phpmyadmin, etc.).

share|improve this answer

... it makes me nervous. Should I be?

It's in your best interest to find out a little bit more about where the wget requests are coming from - this could be a prelude to a scraped version of your content appearing elsewhere (though only a lazy thief would leave wget set as the user agent - wget [...] --user-agent=agent-string will change it).

Definitely check out what an rWHOIS lookup for the IP returns and see if there are any scraper sites associated with it, consider limiting simultaneous requests, and get in touch with your lawyer (or just send a DMCA yourself) if copyrights are being infringed.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.