3

I am implementing a blacklisting system on my website that monitors contact forms for suspicious usage (both spam content and excessive frequency).

When I find somebody / robot that meets my criteria for blacklisting, I want to send them to my DB as a blacklisted entity.

My question is, should I blacklist them as an IP or as a domain?

As far as I can see, blacklisting an IP is going to be far more effective, because I allow people to enter their email address in the form, and they can easily just change their domain on a regular basis.

However, the downside is that if I blacklist an IP, I could potentially be blacklisting a large group of people who share an IP, when only one person is bad (ie - college campuses, coffee shops, etc).

Is there a solution I'm missing?

3
  • ... and blacklisting a full domain would not affect as many people, potentially? Help me understand.
    – jldupont
    Nov 18, 2009 at 18:23
  • The IP you blacklist could be the only public IP if the network is using NAT...
    – OMG Ponies
    Nov 18, 2009 at 18:29
  • @jldupont - well, i can be picky on the domains that i don't blacklist. example, i don't have to blacklist hotmail.com. but you do bring up a good perspective that i hadn't considered as much. Nov 18, 2009 at 18:52

5 Answers 5

8

The easiest approach would be to time limit the blacklisting.
The first time an IP is put into the DB, set the timeout for 3 days or something.
On subequent submissions from that IP:

  • The second time junk comes from an IP set the timeout for 2 weeks.
  • The third time junk comes from that same IP set the timeout for Permanent (0)

  • Or something similar to that, SMTP Greylisting works something like this, and it's quite effective.

    3

    I realize I'm adding to this discussion a little late, but I've taken this idea a step farther and thought my technique might be useful to others. I automatically block the evil-doers, but allow them to unblock themselves. That way, if a shared (or NATted) IP gets blocked, it's not permanent. Here's how:

    I set up a default (scripted) page in one or more subdirectories (folders) blocked in robots.txt. That page, if loaded by a misbehaving robot -- or a snooping human -- adds their IP address to a blocked list. But I have a 403 ("not authorized") error-handler that redirects these blocked IPs to a page explaing what's going on and containing a captcha that a human can use to unblock the IP. That way, if an IP is blocked because one person used it one time for a bad purpose, the next person to get that IP won't be permanently blocked -- just inconvenienced a little. Of course, if a particular IP keeps getting RE-blocked a lot, I can take further steps manually to address that.

    Here is the logic:

    1. If IP not blocked, allow access normally.
    2. If visitor navigates to forbidden area, block their IP.
    3. If IP is blocked, redirect all access to the "unblock" form containing the captcha.
    4. If user manually enters proper captcha, remove the IP from the blocked list (and log that fact).
    5. Rinse, lather, REPEAT above steps for further accesses.

    That's it! One script file to handle the block notice and unblock captcha submission. One entry (minimum) in the robots.txt file. One 403 redirection in the htaccess file.

    Hope this helps ... somebody. It works very well for me. I'm using it on multiple sites.

    2

    Hmmm...it depends on what kind of backend is used on the website, for an example, there's a Bad Behavior plugin that is used on Wordpress installations which does the job of blacklisting. It has a flexibility in that you can blacklist by ip, domain, or user agent string. Furthermore, I am wondering and assuming you have captcha in place?

    1
    • 2
      By the way, forgot to mention the website for Bad Behavior... bad-behavior.ioerror.us
      – tommieb75
      Nov 18, 2009 at 18:34
    2

    Definitely ban an IP. This doesn't solve the problem 100%, but there can be multiple IPs per domain, and certainly some domains you wouldn't want to ban (comcast,etc).

    You're much safer to do it on an IP basis, and you have the same showing up from one subnet:

    192.168.1.10
    192.168.1.25
    192.168.1.30
    

    Then you want to ban by range:

    192.168.1.x
    

    I wouldn't worry too much about blocking lots of people out, it's usually pretty widespread, and you'll find most of your spammers coming from one or two Eastern European countries anyway.

    1

    Well, i guess by domain you mean blacklisting the email address? as blacklisting a whole domain could keep out a LOT more users than blacklisting an ip.

    i think i'd go with the ip, though I think you're adressing this from the wrong point.

    To keep robots/spambots from posting, use a captcha, for instance recaptcha is quite good (and you help to digitize old books by using it).

    To keep bad users out, well, an IP can be quite easily changed, as most people don't have a fixed ip (a fixed ip usually costs extra), so they just reconnect and are good to go untill you spot them again. It's actually quite hard to really keep "evil" users out, though the trick seems to be to not let the banned user know that they're banned, so have the contact form behave as if they weren't, that way it takes them longer to figure it out and change their email/ip. But yeah, why either/or if you could ban their ip AS WELL as their email address?

    2
    • yeah thats definitely a strong product...ive been using recaptcha...im just interested in opening up the system a little and exploring how to make the system strong on my end so that i don't have to tax my users as much.. Nov 18, 2009 at 19:10
    • another, though much more complex, solution could be implementing something like email spam filters, that analyse the words in the contact form and don't let users submit spam comments. After an Initial period of flagging bad ones, the system would be pretty autonomous and you'd just have to flag the ones that get through. it might be overkill for your application, bit It shouldn't be too hard to implement a simple bayesian spam filter. Though I'd only reccomend it to you if you have a decent understanding of math :)
      – Zenon
      Nov 18, 2009 at 19:36

    You must log in to answer this question.

    Not the answer you're looking for? Browse other questions tagged .