2

we've just changed our logging application on our AS/400 partitions and are now using a product called PowerTech SIEM agent for IBM I

The tool used to format messages using the old syslog convention and is apparently now capable of sending IETF messages (RFC 5424), however for some reason our Syslog-NG server is not able to process them, as if the format was not correct.

here is an example error we receive :

Feb  8 15:53:49 hostname syslog-ng[1489874]: Error processing log message: <38>1 2021-02-08T15:53:34.918160+1:00 HOSTNAME.fqn.here - - - - CEF:0!Powertech!SIEM Agent!4.2!AUDIT!TCD0003!6!src=0.0.0.0 dst=1.2.3.4 reason=Command executed msg=A command (WRKPTSA) was run by 12345/T555500/PC01AFB0

the event itself is :

<38>1 2021-02-08T15:53:34.918160+1:00 HOSTNAME.fqn.here - - - - CEF:0!Powertech!SIEM Agent!4.2!AUDIT!TCD0003!6!src=0.0.0.0 dst=1.2.3.4 reason=Command executed msg=A command (WRKPTSA) was run by 12345/T555500/PC01AFB0

Looking at the RFC, it looks like the contents are OK. they should respect :

<priority>VERSION ISOTIMESTAMP HOSTNAME APPLICATION PID MESSAGEID STRUCTURED-DATA MSG

in my case, i have :

<priority> is "38"
VERSION is "1"
ISOTIMESTAMP is "2021-02-08T15:53:34.918160+1:00"
HOSTNAME is "HOSTNAME.fqn.here"
APPLICATION is unknown (-)
PID  is unknown (-)
MESSAGEID  is unknown (-)
STRUCTURED-DATA is unknown (-)
MSG is "CEF:0!Powertech!SIEM Agent!4.2!AUDIT!TCD0003!6!src=0.0.0.0 dst=1.2.3.4 reason=Command executed msg=A command (WRKPTSA) was run by 12345/T555500/PC01AFB0"

we run syslog-NG 3.8

note that i did replace the hostname & IP on purpose, but kept the case.

i have tried multiple things in the syslog-ng configuration (in the source mainly), such as switching from the network() to the syslog() driver (also specified flags(syslog-protocol)), etc. but nothing changed), few examples of what i tried in the source :

source s_ISERIES        { syslog(transport("tcp") ip(0.0.0.0) port(51513)); };
source s_ISERIES        { network(transport("tcp") ip(0.0.0.0) port(51513) flags(syslog-protocol)); };

if i do not use syslog() or the syslog-protocol and use the basic network() driver without flags, the message gets "rewritten" (syslog-NG adds its own header and adds a timestamp and hostname at the beginning of the message, which i do not want).

is there a way to fix that ? do you spot any mistake in the event format ? and if not, is there a way to tell syslog-NG to process the "raw" message (take it and send it where i want without doing further modification of it).

thanks !

edit

after restarting and doing multiple checks, now we just don't have the error anymore. but logs aren't processed at all.

the whole "logging" configuration is quite simple, we've got :

the source defined like this :

source s_ISERIES        { syslog(transport("tcp") ip(0.0.0.0) port(51513)); }; 

the logging config defined like this :

log {
  source(s_ISERIES);
  destination(d_LOCALCOPY_ISERIES);
  destination(d_SPLUNK_ISERIES);
};

destinations defined like this :

destination d_LOCALCOPY_ISERIES   { file("/app/syslog-ng/logs/${YEAR}/${MONTH}/${DAY}/iseries/${HOST}.log" create-dirs(yes)); };

destination d_SPLUNK_ISERIES    { syslog("splunk-host.here.net" port(51513) send-time-zone("UTC")); }; 

and i can't see any message leaving the syslog-NG host when tcpdump'ing (i see them incoming, but not leaving the host). also, no local copy is created. so i "believe" syslog-NG is receiving the message but is unable to parse it for some reason. i tried the flags(no-parse) but it won't work either, as if it was not able to recognize the message at all on the "source" defined.

1 Answer 1

1

As far as I can tell, IBM has never tested their RFC5424 exporter because their timestamp format is garbage and won't work with all too many exporters.

Per RFC5424 (emphasis mine):

TIMESTAMP = NILVALUE / FULL-DATE "T" FULL-TIME

FULL-DATE = DATE-FULLYEAR "-" DATE-MONTH "-" DATE-MDAY

DATE-FULLYEAR = 4DIGIT

DATE-MONTH = 2DIGIT ; 01-12

DATE-MDAY = 2DIGIT ; 01-28, 01-29, 01-30, 01-31 based on month/year

FULL-TIME = PARTIAL-TIME TIME-OFFSET

PARTIAL-TIME = TIME-HOUR ":" TIME-MINUTE ":" TIME-SECOND [TIME-SECFRAC]

TIME-HOUR = 2DIGIT ; 00-23

TIME-MINUTE = 2DIGIT ; 00-59

TIME-SECOND = 2DIGIT ; 00-59

TIME-SECFRAC = "." 1*6DIGIT

TIME-OFFSET = "Z" / TIME-NUMOFFSET

TIME-NUMOFFSET = ("+" / "-") TIME-HOUR ":" TIME-MINUTE

Once you spot it it's actually quite clear in what you get since there's clearly a space included in your timestamp that shouldn't be:

ISOTIMESTAMP is "2021-02-08T15:53:34.918160+1:00 "

2
  • mmh there is no space, it's an issue with the quote i put, there is a space between the timestamp and the hostname. i've just done another tcpdump and i get : "<37>1 2021-10-20T23:38:24.110176+2:00 hostname.here.net - - - - CEF:0!Powertech!SIEM Agent!4.3!AUDIT!TAD0015!5!src=1.1.1.1 dst=2.2.2.2 reason=Auditing setting changed msg=Auditing was changed to *CHANGE for bytestream file *N/*N using the CHGAUD command". But since i restarted the daemon i even get something weirder, no error messages anymore, Syslog-NG is not able to process the message at all, as if the source wasn't working.no errors
    – olivierg
    Oct 21, 2021 at 8:11
  • The timestamp still isn't right. Can you try changing the timezone of the IBM software to something in the 10+ difference with UTC (±1[012]) so the ISOTIMESTAMP conforms to RFC5424, and see if it works? Oct 21, 2021 at 22:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.