2

we've just changed our logging application on our AS/400 partitions and are now using a product called PowerTech SIEM agent for IBM I

The tool used to format messages using the old syslog convention and is apparently now capable of sending IETF messages (RFC 5424), however for some reason our Syslog-NG server is not able to process them, as if the format was not correct.

here is an example error we receive :

Feb  8 15:53:49 hostname syslog-ng[1489874]: Error processing log message: <38>1 2021-02-08T15:53:34.918160+1:00 HOSTNAME.fqn.here - - - - CEF:0!Powertech!SIEM Agent!4.2!AUDIT!TCD0003!6!src=0.0.0.0 dst=1.2.3.4 reason=Command executed msg=A command (WRKPTSA) was run by 12345/T555500/PC01AFB0

the event itself is :

<38>1 2021-02-08T15:53:34.918160+1:00 HOSTNAME.fqn.here - - - - CEF:0!Powertech!SIEM Agent!4.2!AUDIT!TCD0003!6!src=0.0.0.0 dst=1.2.3.4 reason=Command executed msg=A command (WRKPTSA) was run by 12345/T555500/PC01AFB0

Looking at the RFC, it looks like the contents are OK. they should respect :

<priority>VERSION ISOTIMESTAMP HOSTNAME APPLICATION PID MESSAGEID STRUCTURED-DATA MSG

in my case, i have :

<priority> is "38"
VERSION is "1"
ISOTIMESTAMP is "2021-02-08T15:53:34.918160+1:00"
HOSTNAME is "HOSTNAME.fqn.here"
APPLICATION is unknown (-)
PID  is unknown (-)
MESSAGEID  is unknown (-)
STRUCTURED-DATA is unknown (-)
MSG is "CEF:0!Powertech!SIEM Agent!4.2!AUDIT!TCD0003!6!src=0.0.0.0 dst=1.2.3.4 reason=Command executed msg=A command (WRKPTSA) was run by 12345/T555500/PC01AFB0"

we run syslog-NG 3.8

note that i did replace the hostname & IP on purpose, but kept the case.

i have tried multiple things in the syslog-ng configuration (in the source mainly), such as switching from the network() to the syslog() driver (also specified flags(syslog-protocol)), etc. but nothing changed), few examples of what i tried in the source :

source s_ISERIES        { syslog(transport("tcp") ip(0.0.0.0) port(51513)); };
source s_ISERIES        { network(transport("tcp") ip(0.0.0.0) port(51513) flags(syslog-protocol)); };

if i do not use syslog() or the syslog-protocol and use the basic network() driver without flags, the message gets "rewritten" (syslog-NG adds its own header and adds a timestamp and hostname at the beginning of the message, which i do not want).

is there a way to fix that ? do you spot any mistake in the event format ? and if not, is there a way to tell syslog-NG to process the "raw" message (take it and send it where i want without doing further modification of it).

thanks !

edit

after restarting and doing multiple checks, now we just don't have the error anymore. but logs aren't processed at all.

the whole "logging" configuration is quite simple, we've got :

the source defined like this :

source s_ISERIES        { syslog(transport("tcp") ip(0.0.0.0) port(51513)); }; 

the logging config defined like this :

log {
  source(s_ISERIES);
  destination(d_LOCALCOPY_ISERIES);
  destination(d_SPLUNK_ISERIES);
};

destinations defined like this :

destination d_LOCALCOPY_ISERIES   { file("/app/syslog-ng/logs/${YEAR}/${MONTH}/${DAY}/iseries/${HOST}.log" create-dirs(yes)); };

destination d_SPLUNK_ISERIES    { syslog("splunk-host.here.net" port(51513) send-time-zone("UTC")); }; 

and i can't see any message leaving the syslog-NG host when tcpdump'ing (i see them incoming, but not leaving the host). also, no local copy is created. so i "believe" syslog-NG is receiving the message but is unable to parse it for some reason. i tried the flags(no-parse) but it won't work either, as if it was not able to recognize the message at all on the "source" defined.

1 Answer 1

1

As far as I can tell, IBM has never tested their RFC5424 exporter because their timestamp format is garbage and won't work with all too many exporters.

Per RFC5424 (emphasis mine):

TIMESTAMP = NILVALUE / FULL-DATE "T" FULL-TIME

FULL-DATE = DATE-FULLYEAR "-" DATE-MONTH "-" DATE-MDAY

DATE-FULLYEAR = 4DIGIT

DATE-MONTH = 2DIGIT ; 01-12

DATE-MDAY = 2DIGIT ; 01-28, 01-29, 01-30, 01-31 based on month/year

FULL-TIME = PARTIAL-TIME TIME-OFFSET

PARTIAL-TIME = TIME-HOUR ":" TIME-MINUTE ":" TIME-SECOND [TIME-SECFRAC]

TIME-HOUR = 2DIGIT ; 00-23

TIME-MINUTE = 2DIGIT ; 00-59

TIME-SECOND = 2DIGIT ; 00-59

TIME-SECFRAC = "." 1*6DIGIT

TIME-OFFSET = "Z" / TIME-NUMOFFSET

TIME-NUMOFFSET = ("+" / "-") TIME-HOUR ":" TIME-MINUTE

Once you spot it it's actually quite clear in what you get since there's clearly a space included in your timestamp that shouldn't be:

ISOTIMESTAMP is "2021-02-08T15:53:34.918160+1:00 "

2
  • mmh there is no space, it's an issue with the quote i put, there is a space between the timestamp and the hostname. i've just done another tcpdump and i get : "<37>1 2021-10-20T23:38:24.110176+2:00 hostname.here.net - - - - CEF:0!Powertech!SIEM Agent!4.3!AUDIT!TAD0015!5!src=1.1.1.1 dst=2.2.2.2 reason=Auditing setting changed msg=Auditing was changed to *CHANGE for bytestream file *N/*N using the CHGAUD command". But since i restarted the daemon i even get something weirder, no error messages anymore, Syslog-NG is not able to process the message at all, as if the source wasn't working.no errors
    – olivierg
    Oct 21, 2021 at 8:11
  • The timestamp still isn't right. Can you try changing the timezone of the IBM software to something in the 10+ difference with UTC (±1[012]) so the ISOTIMESTAMP conforms to RFC5424, and see if it works? Oct 21, 2021 at 22:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .