We've been using syslog-ng to log our postgres logs. On top of it we run logrotate every 5 minutes which also gzips the files. Recently, we've noted that these postgres logs contain a lot of null characters at the beginning of the file. We later realized that the space occupied by the null characters are equal to the previous size of the file.

Researching (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=733856) on why sparsing has been taking place we realized that the problem lies with syslog-ng not being able to write the logs from the beginning. We tried writing a postrotate script which would kill -HUP syslog-ng. We tried that, but in vain. Many have sorted out this problem by setting the o_append bit. Does anyone know how to set the o_append bit for syslog-ng so that it does not write from the last maintained write head and starts from the beginning of the file?

  • Are you sure those postgresql logs are written by syslog-ng and not directly by postgresql itself? I guess you're using copytruncate, perhaps you shouldn't. – wurtel Mar 20 '15 at 10:32
  • Hey Wurtel, Yes - syslog-ng is doing the logging bit. Are you saying - I'm not supposed to use copytruncate in conjunction with syslog-ng? – Rohit Nair Mar 23 '15 at 9:47
  • copytruncate has a lot of overhead compared to simply renaming the files. Also apparently you need to restart syslog-ng, it has no mechanism for reopening the files, so a SIGHUP won't be enough (that only makes it reread the config). – wurtel Mar 23 '15 at 13:01

Assuming you're talking about /var/log/postgresql/postgresql--main.log: syslog is not your point of control. As wurtel points out, these logs do not come via syslog, but are the stdout/stderr of the postgres process

This is why the standard logrotate for postgres has to be a copytruncate, i.e. copy the file then truncate it to zero bytes. The redirected output will then start writing from the beginning of the file again (at least that's what happen on a standard out-of-the-box system).

Oddly though, if your logrotate isn't copytruncate, then I would expect that after rotate the output would go to the .1 file. Can you share the logrotate config you're using please? I suspect there's a clue in there somewhere.

  • Thank you for your response - This is how our logrotate file looks like : /var/log/pgsql/pgsql.log{ size=500M rotate 2020 copytruncate compress missingok } – Rohit Nair Mar 25 '15 at 13:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.