1

I recently noticed a huge load on my main production server, after some log parsing i stumbled upon a large number of those user agents originate from about 20 different IPs:

173.244.182.194 - - [07/May/2013:16:26:17 +0200] "GET / HTTP/1.1" 302 490 "-" "Java/1.6.0_26" 173.244.182.194 - - [07/May/2013:16:26:18 +0200] "GET / HTTP/1.1" 200 17376 "-" "Java/1.6.0_26"

Guessing it was some kind of screen scraping tool, i managed to block those user agents with fail2ban.

Now i'm puzzled, once the ban time has expired, the requests from the IPs start over and over again. I'm currently blocking about 10 IPs by day.

What is it? The IPs originate from servers and from individuals users, it is a normal event (like those infected servers which constantly try to log with SSH) should i worry about my server security?

2 Answers 2

2

That User-Agent is the default when Java makes an HTTP request. It indicates that it was written by a programmer too lazy to be bothered setting a custom User-Agent.

Even if it's a legitimate bot, it's probably not well written. I would be comfortable blocking it.

I have seen the same IP address (and several others nearby) in my own web logs retrieving very spammy looking URLs such as /blog/there-are-some-diablo-3-gold-players.html and also generic URLs with a spammy looking Referer: header that makes it look like they just did a search for porn.

I have also seen it sending requests like POST /action/sign_in which no legitimate crawling bot should do. It also changes its User-Agent through a short list of normal (if slightly old) browsers.

1
  • interesting, regarding the frequency of the requests and the targeted web pages i first thought of a large scale screen scraping operation because my main server hold a large directory website. Now i think it could just be some kind of random data mining or security fault scanning or maybe some email collecting May 8, 2013 at 16:43
0

Yes you should be concerned about your server security. I would suggest to continue blocking those IP adresses.

If you are worried about ssh login in particular, you could use a tool like DenyHosts. Here's an HowTo.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .