Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Scenario: I am running a cluster of machines. Each machine runs various python programs with a unique (across the cluster), but dynamically set, ID.

Right now, they are all logging locally. So, I might have logs that look like:

process_5.log
process_6.log

for processes that had ID's 5 and 6.

Another machine may have:

process_20.log
process_25.log

I wish to forward these logs to a logserver running rsyslogd. Python's logging facility has a nice syslog handler, so I understand how I could connect to the remote server. What I haven't figured out is how to use templating/DynFile to maintain log separation.

e.g. on the logserver, I will want to see:

process_5.log
process_6.log
process_20.log
process_25.log

which correspond to the logs of the same name on the sending machine.

Is there a way to pull this off with rsyslogd templating?

share|improve this question
add comment

2 Answers

As the user quanta posted, their template should work. Here is mine:

$template DynFile, "/var/log/syslog/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/%syslogfacility-text%.log"
*.info;mail.none;authpriv.none;cron.none            ?DynFile
authpriv.*                                          ?DynFile
mail.*                                              ?DynFile
cron.*                                              ?DynFile
uucp,news.crit                                      ?DynFile

This will make logs for each app or facility go into its own file. Each client's log on the server will appear just like on the client.

Notice the %syslogfacility-text%. You can substitute this part to various other properties per the Property Replacer documentation.

share|improve this answer
add comment

Each machine runs various python programs with a unique (across the cluster), but dynamically set, ID.

Does the process_$ID is also the program name? And if so, try this:

$template DynaFile,"/var/log/rsyslog/python_apps/%PROGRAMNAME%.log"
if $programname startswith 'process' then -?DynaFile

You can put it in separate folder with %HOSTNAME% if you want.

PS: I haven't tested. Let me know if it works for you.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.