0

I am contacting you regarding an issue I have with syslog-NG. some of our devices (mainly HP switches and SANs) are sending syslog messages that do not respect the syslog RFC 5424 it seems.

to give you a few examples :

if I sniff the network interface of the server, we can see these "wrong" messages like that (as you can see, after the PRI, we can see the PROGRAM, then timezone and fields separated with commas. in short, mixed fields, missing fields etc. not a standard syslog message) :

<190>raslogd: 2017/03/08-16:03:20, [SEC-1203], 53642, WWN 10:00:50:eb:1a:6c:21:38 | FID 128, INFO, cswc-mo8x-SAN01, Login information: Login successful via TELNET/SSH/RSH. IP Addr: 1.2.3.4

therefore, I am currently unable to filter these messages. I tried to define a regular filter and try to print fields such as MSGHDR, MSG, PRI etc.. but couldn't see anything.

the ONLY WAY of having this message filtered is by just defining the external interface, and a file as output, without any template.

e.g if I do a filter like this :

destination d_INCOMING_ALL   { file("/app/syslog-ng/logs/incoming_all.log"); };

log {
  source(s_EXTERNAL);
  destination(d_INCOMING_ALL);
};

I can see messages in the log file, but formatted, somehow (I suppose syslog-NG reformats them) :

[root@xxxxxxxxxxxx logs]# grep -i cswc incoming_all.log
Mar  9 09:44:20 cswc-mo8x-hpsan01 raslogd: 2017/03/09-08:34:50, [SEC-1203], 53647, WWN 10:00:50:eb:1a:6c:21:38 | FID 128, INFO, cswc-mo8x-SAN01, Login information: Login successful via TELNET/SSH/RSH. IP Addr: 1.2.3.4
[root@xm1p1034vmo logs]#

the problem is that I cannot filter these messages like that (we receive logs from more than 1000 devices) there, i need to filter messages coming from these devices ! and the only way I can do it is on the hostname (cswc-) or program name (raslogd)

so I tried to display the fields by adding a template to that file, example :

destination d_test { 
    file ("/app/syslog-ng/logs/test_olivier.log" 
    template("pri=${priority} date=${ISODATE} host=${HOST} program=${PROGRAM} message=${MSG}\n") 
    ); 
};

but nothing works, nothing is displayed. I tried all fields, MSG, MESSAGE, MSGHDR etc.. can't manage do display ANYTHING. the only working thing is the parsing without filters or templates.

naturally, if I tred all kind of filters, like these below, it does not work (as fields are mixed) :

filter f_is_SAN     {
    host("cswc.*" flags(ignore-case));
};

same for :

filter f_is_SAN     {
    match(".*cswc.*" flags(ignore-case));
};

any hints on how to create filters for these messages coming from these devices (on hostname or programname) ?

thanks regards,

1

Actually, syslog-ng is not reformatting them, it is adding a proper syslog header:

Mar 9 09:44:20 cswc-mo8x-hpsan01 raslogd: 2017/03/09-08:34:50, [SEC-1203], 53647, WWN 10:00:50:eb:1a:6c:21:38 | FID 128, INFO, cswc-mo8x-SAN01, Login information: Login successful via TELNET/SSH/RSH. IP Addr: 1.2.3.4

The following is added by syslog:

Mar 9 09:44:20 cswc-mo8x-hpsan01

The rest is the original message:

raslogd: 2017/03/09-08:34:50, [SEC-1203], 53647, WWN 10:00:50:eb:1a:6c:21:38 | FID 128, INFO, cswc-mo8x-SAN01, Login information: Login successful via TELNET/SSH/RSH. IP Addr: 1.2.3.4

So syslog-ng is agreeing with you that these messages are not RFC-compliant.

In terms of setting up filters, this means you do not have a lot to play with.

I am personally more used to rsyslog (where you could inspect the message with regex, e.g. looking for raslogd - although that is not necessarily efficient).

However, the header syslog-ng is able to add gives you a couple of things to go on:

  • it seems syslog-ng is able to report a hostname, which I would tend to assume is resolved from the source IP. So you at very least can filter on source ip, using either host or netmask:

    filter hp_hosts { (host(192.168.0.25)) };

  • your current host filter is failing because there is no HOST field in the messages - the documents for the host filter state:

    That is, syslog-ng OSE will compare the filter expression to the content of the ${HOST} macro.

  • you do however have a hostname (at least in your example) within the message, so you could consider using a message filter.

That being said - to me, if it was possible to get a list of the source IPs of all these devices (and I can only hope for your sake that is possible), then I would use the netmask filter - it seems likely to be more efficient than the other options.

  • thanks for your detailed answer. filtering on the host does not work (whether it's an IP or a hostname) the only way to filter the message is by adding the "flags(no-parse)" to the source (UDP), but it screws up all my other filters naturally, so I can't do that :( any other idea ? – olivierg Mar 24 '17 at 16:00
  • Have you tried using the netmask filter? – iwaseatenbyagrue Mar 24 '17 at 16:05
  • not yet, I will try, however all my machines are on the same subnet, so.. I can try to specify the IP on a /32 and see... I will try and let you know – olivierg Mar 24 '17 at 17:07
  • I tried every kind of filter, including host, netmask etc.. and nothing works. the only way to make it work is by changing the source and adding no-parse flag, but it screws up all my other filters/logs. frustrating :( I wish there was a generic filter, as I can process the message if it's not fitered – olivierg Mar 27 '17 at 13:44
  • 1
    Have you considered looking at rsyslog in place of syslog-ng? The sort of filter you wanted could be written as :fromhost-ip,isequal,"1.1.1.1" (or, in more recent versions, the last argument can IIRC be an array of terms). rsyslog has its own quirks, but this one specific problems seems like it could be handled by rsyslog (I have some vague memory of having dealt with this issue in the past, although I don't remember the exact details - but it certainly involved rsyslog). – iwaseatenbyagrue Mar 27 '17 at 13:49
1

mmh I found a solution which seem to be working

I am reusing the "local copy" destination as a source

e.g :

I have a regular log filter keeping a copy of everything :

destination d_INCOMING_ALL       { file("/app/syslog-ng/logs/incoming_all.log"); };
log {
  source(s_EXTERNAL);
  destination(d_INCOMING_ALL); 
};

the message is kept in that "local copy" being reformatted. I can then re-read from this file in a source and apply the filters to it. it's dirty, but it works :

source s_NON_RFC_COMPLIANT {   # new source
    file("/app/syslog-ng/logs/incoming_all.log" follow-freq(10));
};

....

destination d_INCOMING_ALL {
    file("/app/syslog-ng/logs/incoming_all.log");    # old destination where we keep a local copy of EVERYTHING
};

....

log {      # local copy of everything
  source(s_EXTERNAL);
  destination(d_INCOMING_ALL);
};

....

other filters here

....

filter f_is_Network     {
  host("cswc.*" flags(ignore-case))       # we want to filter on hosts called cswc-something or program raslogd
  or program("raslogd" flags(ignore-case));
};

log {    # new source for HP stuff here
  source(s_NON_RFC_COMPLIANT);
  filter(f_is_Network);              # add the filters you need
  destination(<your_final_destination_here>);
  flags(final);
};

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.