"deny all" for all files in a directory

Question

My server is getting hit hard by bots, etc., looking for various files in /cgi-bin/, like search.cgi, YaBB.pl, gitweb.perl, perl, gitweb.pl, htsearch, and on and on.

I want to deny all access (send a 403 Forbidden header) for all possible file names and extensions in /cgi-bin/ . (This is a shared server, so I only have access to .htaccess, not vhosts or httpd).

In an .htaccess in cgi-bin I've tried

deny from all

as well as

<FilesMatch "\.cgi?$">
        Order Allow,Deny
        Allow from All
</FilesMatch>

for just .cgi extensions, with no luck for either.

How can I deny access for everything in /cgi-bin/?

Do I need to list all file names and extensions in a <FilesMatch directive?

Answer 1

Do you know the ip or user-agent which is cause of high traffic?

<Directory /cgi-bin>
  order deny,allow
  deny from all
</Directory>

Alt,

RewriteRule   "cgi(.*)"   "go_away"

Answer 2

There is lot of way to achieve this :

1. Using Directory (Apache 2.2 version):

   <Directory /var/www/cgi-bin/>
   order deny,allow
   deny from all
   </Directory>
   

2. Using rewrite condition:

   RewriteCond %{REQUEST_URI} ^/cgi-bin
   RewriteRule .* - [END,R=406]
   

you can change 406 by any code you want as 403 for forbiden, you can find here the list of status code : https://en.wikipedia.org/wiki/List_of_HTTP_status_codes

And you can even extend the rewrite condition to cover more bot:

RewriteCond %{HTTP_USER_AGENT} ^-? [OR]
RewriteCond %{REQUEST_URI} ^/cgi [OR,NC]
RewriteCond %{REQUEST_METHOD} !^(GET|HEAD|POST)$ [OR]
RewriteCond %{REQUEST_URI} !^/
RewriteRule .* - [END,R=406]

thoose rules prevent from empty useragent, request starting by /cgi, limit method to GET/HEAD/POST, request not starting by /

3. Using the more logical way:

Just move or remove cgi-bin from your web folder if you don't use them.