.htaccess regular expressions for optimal performance?

Question

In my .htaccess file I'm trying to block some bots using:

RewriteCond %{HTTP_USER_AGENT} ^badbot [NC]

But, I want to change these conditions to something more inclusive like:

Method #1: RewriteCond %{HTTP_USER_AGENT} .*badbot [NC,OR]

OR

Method #2: RewriteCond %{HTTP_USER_AGENT} ^.*badbot [NC,OR]

OR

Method #3: RewriteCond %{HTTP_USER_AGENT} badbot [NC,OR]

The question is, which one of those 3 methods would be best for server performance?

Thanks!

Answer 1

As far as I'm aware, there is little performance difference between these. Logically, the /.*badbot/ and /badbot/ should be equivalent, and as regular expressions are parsed from left to right I would also have thought the ^ made no difference.

I did test these with Perl (which may use a slightly different regex engine to mod_rewrite) and it did show /badbot/ to be quite a bit quicker than the others, but this could be down to the strings I was using. Also, bearing in mind that USER_AGENT has a maximum length of 200, this is probably unnecessary optimization. You'd get more optimization by placing these inside the httpd.conf instead of .htaccess.

The results I got from testing are:
[james@aladdin ~]$ time perl -e 'my $x = "bad" x 99999999; $x .= "badbot"; print "true" if $x =~ /^.*badbot/;'
true
real 0m2.575s
user 0m1.847s
sys 0m0.712s
[james@aladdin ~]$ time perl -e 'my $x = "bad" x 99999999; $x .= "badbot"; print "true" if $x =~ /badbot/;'
true
real 0m2.110s
user 0m1.378s
sys 0m0.691s
[james@aladdin ~]$ time perl -e 'my $x = "bad" x 99999999; $x .= "badbot"; print "true" if $x =~ /.*badbot/;'
true
real 0m2.522s
user 0m1.866s
sys 0m0.647s

Answer 2

If your looking for optimal performance... Firstly have Apache ignore/skip looking for .ht files.. Disable .htaccess.. :D

AllowOverride None

And put your stuff/rewrites/etc directly in each vhost..

And then choose 1,2 or 3 :D

Then you have already taken a step towards "faster"

Hope this helps.. :D