How to send a systemd ssh error so that it's picked by fail2ban

Question

I have a setup which uses ansible and vagrant to test a custom role which installs and configures fail2ban.

I have written myself a simple test playbook which uses the fail2ban role and applies the necessary configuration as expected. My intention, however, was to also write a few lines which do an integration test such as simulate a few "failed login" attempts.

From what I saw in How can I send a message to the systemd journal from the command line?, one should be able to do something like this:

echo "Failed password for root from 1.2.3.4 port 35720 ssh2" | systemd-cat -t sshd -p err

This indeed shows up in the journalctl -f logs, but it somehow doesn't end up in fail2ban at all:

fail2ban-client status sshd
Status for the jail: sshd
|- Filter
|  |- Currently failed: 1
|  |- Total failed: 7
|  `- Journal matches:  _SYSTEMD_UNIT=sshd.service + _COMM=sshd
`- Actions
   |- Currently banned: 0
   |- Total banned: 1
   `- Banned IP list:   

The stats from above are from my own login failures.

I also tried with different log levels (i.e. crit, err, info, warning, etc), but without any luck. Is there anything I'm doing wrong here?

Answer 1

I have managed to figure this out after countless hours of reading and reading through a bunch of man documentation:

1. logger
2. systemd.journal-fields

Once you go through a bunch of pages in the docs you would find out that you can do something like the following:

journalctl --facility=auth --since -5m --output=export

This shows the raw message with all of it's metadata. The properties which are starting with _ are considered "trusted" and cannot be manipulated (they are appointed by syslog or journald).

Here's an example message:

_SYSTEMD_SLICE=system.slice
_CAP_EFFECTIVE=3fffffffff
SYSLOG_FACILITY=4
_TRANSPORT=syslog
SYSLOG_IDENTIFIER=sshd
_COMM=sshd
_EXE=/usr/sbin/sshd
_SYSTEMD_CGROUP=/system.slice/ssh.service
_SYSTEMD_UNIT=ssh.service
_SYSTEMD_INVOCATION_ID=f29bc3ffcdce4a2dbeb18fe53ac7fea9
_CMDLINE=sshd: root [priv]
SYSLOG_PID=7021
SYSLOG_TIMESTAMP=Aug  6 11:18:42 
_PID=7021
MESSAGE=Connection closed by authenticating user root 192.168.121.1 port 54666 [preauth]
_SOURCE_REALTIME_TIMESTAMP=1659784722477758

In fail2ban the journal match filter is _SYSTEMD_UNIT=sshd.service + _COMM=sshd. So, it is not possible for me to simulate login failures. Therefore I had to change the filter to be journalmatch = SYSLOG_IDENTIFIER=sshd + SYSLOG_FACILITY=4 (located in filers/sshd.conf).

Once I did that, I could then easily simulate a login failure message with the following command:

logger --journald <<END
SYSLOG_FACILITY=4
SYSLOG_IDENTIFIER=sshd
MESSAGE=Failed password for root from 172.31.255.254 port 54666 ssh2
PRIORITY=6
END

And voilà - integration test completed.

Every 2.0s: fail2ban-client status sshd                                                                  localhost: Sat Aug  6 12:00:22 2022

Status for the jail: sshd
|- Filter
|  |- Currently failed: 0
|  |- Total failed:     1110
|  `- Journal matches:  SYSLOG_IDENTIFIER=sshd + SYSLOG_FACILITY=4
`- Actions
   |- Currently banned: 1
   |- Total banned:     8
   `- Banned IP list:   172.31.255.254