3

I must admit, regexs have always been a weak spot for me. I've never sat down and spent enough time learning them to use them effectively. However, I really am at a loss as to why this is not matching. I'm trying to make a custom failregex to stop people trying to exploit any xmlrpc.php bugs (and they are constantly brute-forcing in an attempt to do so).

I am using fail2ban v0.9.3, with Apache 2.4.18 on Ubuntu 16.04.2.

Here is my regex:

<HOST> - - \[\d{2}/\w{3}/\d{4}:\d{2}:\d{2}:\d{2} +\d{4}\] "POST /xmlrpc.php HTTP/1.0" 200

And here is an example line that it isn't matching (IP replaced with 0s):

0.0.0.0 - - [06/Apr/2017:07:45:42 +0000] "POST /xmlrpc.php HTTP/1.0" 200 752 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"

I cannot figure out why it's not matching. As far as I can tell, that should be an exact match. Can anyone shed some light?

Additionally, it would be nice to have the time zone offset match for both a + and a - sign... and I think by replacing HTTP/1.0 with HTTP/1.\d{1} it should match both 1.0 and 1.1, correct?

Thanks in advance!

EDIT: After playing around on the commandline with fail2ban-regex, I've figured out that with a regex of <HOST> - - \[ it will match all lines. However, the second I change it to <HOST> - - \[\d{2} it does not match any lines at all. This doesn't seem to make sense!

EDIT 2: Well, in the meantime, I'm using <HOST> - - \[.* "POST /(xmlrpc|wp-login).php HTTP/1.\d" 200 and that is matching the lines I want. However, I would still love to know why my original one isn't matching, as I'd rather use a more specific regex than the broad one I'm using now.

2

After a couple of iterations with fail2ban-regex, I think what happens is that fail2ban tries to pick the date from the log line itself, before testing it against your filter pattern.

$ line='0.0.0.0 - - [06/Apr/2017:07:45:42 +0000] "POST /xmlrpc.php'

With .* in place of the timestamp, it matches, but as you noticed, explicitly looking for a number at the start of the timestamp doesn't match:

$ ./fail2ban-regex "$line"  '<HOST> - - \[.*\].*POST /xmlrpc' |grep ^Lines
Lines: 1 lines, 0 ignored, 1 matched, 0 missed

$ ./fail2ban-regex "$line"  '<HOST> - - \[\d+.*\].*POST /xmlrpc' |grep ^Lines
Lines: 1 lines, 0 ignored, 0 matched, 1 missed

But check this out:

$ ./fail2ban-regex "$line"  '<HOST> - - \[ \+0000\].*POST /xmlrpc' |grep ^Lines
Lines: 1 lines, 0 ignored, 1 matched, 0 missed

I removed the whole timestamp from the pattern, instead expecting it to be empty (there's just the timezone indicator). It still matches, hinting that fail2ban actually removes the timestamp before testing against the pattern in the configuration.

So, it may be better to just leave that part out of the filter pattern, and match anything in place of the timestamp:

<HOST> - - \[.*?\] "POST /xmlrpc.php HTTP/1.0" 200

As a side note, your original RE has an unquoted + where the timestamp starts, the plus sign means "at least one repetition of the previous element", so you need to escape it.

  • Awesome work!! Thank you so much. Going to implement this now. I hope your answer will help others in my position! – Aurelius Apr 7 '17 at 5:50
  • This is what I'm using now, and it seems to be working well! <HOST> - - \[.*?\] "(GET|POST) /(xmlrpc|wp-login).php HTTP/1.\d" 200 – Aurelius Apr 7 '17 at 5:51
5

Try this regex - I tested it with regexr.com and it is definitely being matched:

<HOST> - - \[\d{2}\/\w{3}\/\d{4}(:\d{2}){3} \+0000\] "POST \/xmlrpc.php HTTP\/1.0" 200

One of the issues was not escaping the slashes - this needs to be done.

EDIT (After I saw your edit): I tried simplifying it a bit and came up with this that matches:

failregex = ^<HOST> - - \[.*\] "POST \/xmlrpc.php.*$

It is not exactly what you wanted but it works

  • The problem is even <HOST> - - \[\d{2} doesn't match. The second I add \d{2} it stops matching. I feel like fail2ban may be using a different regex pattern matching syntax? In which case... dear lord. Where to begin? Their website doesn't have much info that I can find. – Aurelius Apr 6 '17 at 9:10
  • @Aurelius - Check my updated answer. I am not an expert in fail2ban so I cannot explain why the other regex doesn't work – Frederik Nielsen Apr 6 '17 at 9:45
  • That is too funny! About 5 minutes before you edited your post, I came up with the same answer and edited my question :) – Aurelius Apr 6 '17 at 9:45
  • Please note that regexr.com uses the browsers RegExp implementation, and is most probably not the same engine as your fail2ban installation uses. regex101.com at least has some different options for parsing engines. – sshow Apr 6 '17 at 13:40
  • @sshow - Thanks for the info - did not know that! – Frederik Nielsen Apr 6 '17 at 14:17

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.