Capture only certain logs with syslong-NG

Question

I'm new to Syslog-ng. sofar I deployed it and it is running fine pushing some logs to MongoDB. Now, what I really want to do is push some logs (e.g. /var/log/secure) to MongoDb while pushing some other logs to file system and yet ignoring some other set of logs.

can you guys help me define such a configuration?

thanks, Max.

Answer 1

/var/log/secure should be coming from syslog-ng already. As such you just need to redirect where log entries are going.

Try not to think of things in terms of files. /var/log/secure is a file yes, but only because syslog-ng is configured to put log entries there. Log entries come in with all sorts of attributes like the 'facility', the 'priority', the program name, the PID, etc. If I want to, I could tell syslog-ng to route all log entries from the program 'sshd' to /var/log/secure.

So what you need to do is change the criteria of what gets routed to mongodb. Lets pretend your syslog-ng config looks something like this:

filter f_secure {
  facility(auth) or facility(authpriv);
};
destination d_secure {
  file('/var/log/secure');
};
log {
  source(s_local); filter(f_secure); destination(d_secure);
};

filter f_mongodb {
  program('foobar');
};
destination d_mongodb {
  mongodb(
    ...
  );
};
log {
  source(s_local); fitler(f_mongodb); destination(d_mongodb);
};

All you need to do is change the filter for mongodb to include the same things that the filter for /var/log/secure has. Depending on your version of syslog-ng, this could be as simple as changing f_mongodb to the following:

filter f_mongodb {
  filter(f_secure) or program('foobar');
};

All we did was included the definition of f_secure into f_mongodb. Now any changes to f_secure will also affect what goes into mongodb.

However, older versions of syslog-ng you must do the following (as recursive filters were added in a 3.X version I believe):

filter f_mongodb {
  ( facility(auth) or facility(authpriv) ) or program('foobar');
};

This does the exact same thing as our previous example. The only difference is that we replaced filter(f_secure) with the contents of the filter itself. This is all filter(f_secure) does. It just makes your config simpler to maintain so you don't have duplicate config lines everywhere.

Note that the parenthesis around the facility(auth) or facility(authpriv) arent strictly necessary, I just included them as that the logical equivalent of what filter(f_secure) does