Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Are there internet users connecting with HTTP/1.0? Such as over a proxy. Because I'm going to block HTTP/1.0 since most of the bots are using HTTP/1.0.

share|improve this question
    
If you would like specifics on what to do, please add your platform (e.g. linux/apache, or windows/IIS, including version numbers, etc.) –  jmsmcfrlnd Feb 8 '10 at 23:39
    
FreeBSD 6.3-PRERELEASE, Apache 2.0.61, PHP 5.2.5 –  ilhan Feb 8 '10 at 23:54

2 Answers 2

up vote 4 down vote accepted

While it is mostly in use by bots and other crawlers, why not block bots the regular way, by adding/changing the robots.txt file (for the "law-obeying" bots) or by changing your .htaccess file to block the criminals. This site is one of many describing these procedures.

Update: If I were to write a malicious bot today, I'd use http/1.1, just to throw off people blocking 1.0 :).

Second, there may be some legacy browsers that may still use 1.0 (AFAIK all recent browsers use 1.1). These may also include accessibility-type browsers (like Lynx, or blind-support browsers) or browsers on phones/devices that cannot be updated (i.e. my Nokia 6310's browser uses 1.0 - luckily I've switched to a Blackberry). Blocking 1.0 might lose you that type of traffic.

But here's the real easy solution: monitor your logs for a while. If you don't see a lot of 1.0 requests - block it, safely assuming you won't lose too much traffic.

share|improve this answer
    
most of them are using web browsers' user-agent strings. i'm trying to block only the criminal ones. and i have already blocked .ru mail addresses and chinase ip addresses. but the question is about http/1.0 users. I don't care about bots, criminals or not. I might block all of them, except Googlebot. –  ilhan Feb 8 '10 at 23:57
    
Check if my update addresses your comment. –  Traveling Tech Guy Feb 9 '10 at 1:09
    
lynx, links, curl and wget are using HTTP/1.1 by default. May be very old versions are using HTTP/1.0. I do not expect a decent crawler to use HTTP/1.0, because HTTP/1.1 is required for name based virtual hosts. –  Mircea Vutcovici Feb 15 '10 at 21:48
    
I've had a blind colleague who's been using a reader (essentially a box connected to a machine that reads the characters on the screen out loud). It uses a proprietary browser. He has used it since 1994 (it actually uses a pre-USB serial connector) and I very much doubt its browser uses 1.1. –  Traveling Tech Guy Feb 16 '10 at 5:36

Yes, Squid uses HTTP/1.0

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.