Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a forum/groups site that contains a lot of external URLs, sometimes direct download links. I want to protect my visitors from possible attacks from malware sites as they are mot likely to click on these links. CUrrently I implement DBL (spamhaus) but thats not enough. I want to run a background task to check the outgoing links first. I have looked at similar questions in StackOverflow (wrongly posted there) and here but fail to find a question same as mine or a good answer.

People have suggested ClamAV , I don't believe it can detect Web hosted malware sites and its has a lot of missed detection. I have looked at google safe browsing service ( very complicated to implement or maintain plus midway I get lost :S )

I can go for commercial solution, anything to protect the visitors and my site brand. But I would like to hear the opinion of server admins and if anyone has implemented such a service.

My Server is basic CentOS LAMP stack.

thank you very much in advance.

share|improve this question
up vote 0 down vote accepted

I use 3 or 4 external site checking services, in a crontab script. this is written in my lang (tcsh) but easily convertible to bash/sh

I run this once a day.

Possibly a difficult part might be assembling the list of the external sites you link to.

#!/bin/tcsh -f 
# simplistic after the fact check/test of our sites,being possible malware related.
#Mon Sep 20 18:52:15 GMT 2010,dianevm at
# happened once when some bogus advert networks were used for 48 hours :-(
setenv TZ CST6CDT

set LINKS="links -no-references -no-numbering -dump-width 120 -dump "
set TMPF=/tmp/.malware.dmp.$$
#alias DBG 'echo -n DEBUG:; set PAUSE=$<'
alias DBG 'echo -n " "'
set NOW=`date +%T`

alias OKOUT 'set NOW=`date +%T`;printf %-8s \!*;echo " $NOW"'


foreach i ( $SITES2CHECK )

echo ___  $i ___  

printf %-20s GOOGLE 
$LINKS "$i" >! $TMPF

set GOOGLEOK=`grep 'This site is not currently listed as suspicious' $TMPF |wc -l` 
if  ( "$GOOGLEOK" == "1" )then 
    OKOUT ok
   tcsh ~/malwarefail $i GOOGLE $TMPF

printf %-20s SiteAdvisor  
set SITEADVOK=`grep 'tested this site and didn.t find any significant problems.' $TMPF|wc -l` 
set SITEADVUNKNOWN=`grep 'we haven.t tested this one yet.' $TMPF|wc -l` 
if ($SITEADVOK == "1" || "$SITEADVUNKNOWN" == "1") then
    OKOUT  ok  
   tcsh ~/malwarefail $i SITEADV $TMPF

printf %-20s  Norton 
$LINKS "$i"  >! $TMPF
set NORTONOK=`grep 'Norton Safe Web found no issues with this site' $TMPF|wc -l` 
set NORTONUNKNOWN=`grep ' This site has not been tested yet' $TMPF|wc -l` 
if ($NORTONOK == "1" ||$NORTONUNKNOWN  == "1" ) then
    OKOUT  ok  
    tcsh ~/malwarefail $i NORTON $TMPF
printf %-20s  BRWSDEFNDR 
$LINKS "$i/">! $TMPF
set BRWSDEFNDROK=`grep 'Our testing of this site found no dangerous downloads' $TMPF|wc -l` 
set BRWSDEFNDRUNKNOWN=`grep 'Not yet rated' $TMPF|head -1|wc -l` 
#note head added,2  instances 
if ($BRWSDEFNDROK == "1" ||$BRWSDEFNDRUNKNOWN  == "1" ) then
    OKOUT  ok  
    tcsh ~/malwarefail $i BRWSDEFNDR $TMPF
## note malwarefail just emails people the output of the dump files
share|improve this answer
This is a good and usable starting point. Thank you so much for sharing this. I am sure it will help someone else too. With a little bit of coding here and there anyone can adapt this. – Vangel Mar 22 '11 at 7:24

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.