My observation of logrotation is that to rotate any log file, the logrotate process, in the following order
- copies the log file in question (lets call it file1.log) with a new name (adding time stamp or a number to the existing name so that it becomes file1.log-20140513),
- deletes the existing file (file1.log) and creates a new blank log file with the original name (file1.log),
- compresses the rotated file (file1.log-20140513), which creates a new compressed file (file1.log-20140513.gz), if compression option is set,
- deletes the rotated file (file1.log-20140513), and the finally,
- move to the next file to do the same above 4 steps to it.
I am having the following problem with this process:
- My log files are huge in size (more than 10 Gb each) and I have around 42 such log files.
- The processes which write to these files work in sync,
- The DiskIO on the server is good, but still, copying takes time, and compressing also takes time and compression also consumes CPU.
- I want all the newly created log files to have logs starting from same time.
To do that, I want logrotate to move, some thing which the mv command does, which renames the files instead of copying them. As for compression, I can disable that and trigger it via a different script scheduled via cron. But I want logrotate to move the files instead of copying them.
Now, am sure this is some thing the authors of logrotate too would have thought as it obviously saves disk IO and time taken to complete the entire logrotate operation, so I want to why files are being copied instead of being moved or renamed, and how do I achieve this via logrotate.
NOTE: I tried to do it manually, which was, move the file to which a running process was writing too, and create a new blank file with same name and same permissions (which is root, which is also the permission which the process is running with), but after moving and creating a new file, I saw that the process was not writing anything to it, so had to restart the process to make it write to that file. Can any one explain this behaviour too as to why logrotate manages to make the process write to the same file but I'm not able to using the simple steps.