0

We have rsyslog that save logs based on requirement to various directories:

/opt/logs/stats/dc1/stats_YYYY-MM-DD_%somename%.log
/opt/logs/stats/dc2/stats_YYYY-MM-DD_%somename%.log
/opt/logs/events/dc1/events_YYYY-MM-DD_%somename%.log
/opt/logs/events/dc2/events_YYYY-MM-DD_%somename%.log
/opt/logs/security/dc1/security_YYYY-MM-DD_%somename%.log
/opt/logs/security/dc2/security_YYYY-MM-DD_%somename%.log

How to archive these logs weekly ?

I would like them compressed and moved to a different folder:

/mnt/logs_archive/stats/dc1/stats_<YYYY-MM-DD>.tar.gz
/mnt/logs_archive/events/dc1/events_<YYYY-MM-DD>.tar.gz
/mnt/logs_archive/security/dc1/security_<YYYY-MM-DD>.tar.gz
/mnt/logs_archive/stats/dc2/stats_<YYYY-MM-DD>.tar.gz
/mnt/logs_archive/events/dc2/events_<YYYY-MM-DD>.tar.gz
/mnt/logs_archive/security/dc2/security_<YYYY-MM-DD>.tar.gz

Where YYYY-MM-DD is the date of the move that way we know that it shall include last 7 days of logs from the date.

Question:

  1. Is there a way to achieve this in a single/simple config
  2. Will I have to create a new one for log path?
  3. For Certain folders (security events) we are not interested in archiving logs for more than 120 days. can this be achieved too?

My current copy:

copytruncate
compress
dateformat _%Y-%m-%d.
dateext
extension log
olddir /mnt/archive_logs/

/opt/logs/stats/gw_stats*.log {
    rotate 48
    hourly
}

This gives me multiple compressed files

ll /mnt/archive_logs/


stats_YYYY-MM-DD_f2._2021-04-02.log.gz
stats_YYYY-MM-DD_asdas3._2021-04-02.log.gz

can I merge this into one stats_YYYY-MM-DD.gz ?

Also how do i use tar.gz ?

Improve this question
1
  • show us please your current file of log rotate so we can see what you have already been done. you can use one config file to add all paths you need
    – djdomi
    Apr 2, 2021 at 16:53

1 Answer 1

Reset to default
1

You could write a single configuration file with different stanzas, one for each file to be rotated and use the directives dateext, dateformat, olddir, but be carefull with olddir (your /mnt dir suggests that the physical device is different so, probably you should make use of postrotate to move rotated files)

olddir directory

Logs are moved into directory for rotation. The directory must be on the same physical device as the log file being rotated, and is assumed to be relative to the directory holding the log file unless an absolute path name is specified. When this option is used all old versions of the log end up in directory. This option may be overridden by the noolddir option.

Your configuration file would look (assuming the names stats.log and events.log as the ones of log files).

compress
weekly
rotate 120
dateext
dateformat _%Y-%m-%d

/opt/logs/stats/dc1/stats.log {
   olddir /mnt/logs/archive/stats/dc1/
}

/opt/logs/events/dc1/events.log {
   olddir /mnt/logs/archive/events/dc1/
}

...
Improve this answer
9
  • Updating question with copy of what I have for stats there are multiple log files as stats_YYYY-MM-DD_<some_name>.log I assume that I can just * to include everything
    – sarvesh.lad
    Apr 2, 2021 at 18:10
  • Hmm, yes, but be carefull, (serverfault.com/questions/997734/… although using olddir that should not be a problem) and if the names have the date extension, you should not add it.
    – J.M. Robles
    Apr 2, 2021 at 18:17
  • Thanks, so that gets me partially what I wanted, but I was hoping to archive all log files for a given week into one archive preferrbly tar split into 500 mb blocks each
    – sarvesh.lad
    Apr 2, 2021 at 18:35
  • My guess is just move to old dir and and use post script to call tar however I want?
    – sarvesh.lad
    Apr 2, 2021 at 18:37
  • With postrotate you adquire a better control. And the script does not seem to be too difficult
    – J.M. Robles
    Apr 2, 2021 at 18:48

Not the answer you're looking for? Browse other questions tagged .