I have a simple user systemd service configured like so:


ExecStart=/home/mikel/anaconda3/bin/python -u bot.py parameters.json


Essentially, all the service does is run a python script.

However, when I look at the journalctl for the service, I see that instead of logs appearing as things happen in the script, it is all flushed at once when the service is stopped (with the same timestamp for every line).

This suggests that some buffer is occuring somewhere that is preventing it from being written to journalctl immediately. However, even when I turn off Python's internal buffer with -u, the problem still occurs.

One solution I have found is to wrap it in the unbuffer command from the expect package, like so:

ExecStart=/usr/bin/unbuffer /home/mikel/anaconda3/bin/python -u bot.py parameters.json

However, while this works, it means that journalctl shows "unbuffer[PID]" instead of "python[PID]", and I want a solution that doesn't involve installing external packages for what seems like a simple task.

Is there some other way to remove these buffers so that the output is flushed to journalctl as it comes from python?

  • 1
    Can you give more details on what bot.py is doing? Is it calling external programs? That could explain why the output is unbuffered on those and why unbuffer would fix it...
    – filbranden
    Apr 24 '18 at 17:09
  • 2
    Maybe not exactly what you want... But look at SyslogIdentifier= config in systemd. You could use something like SyslogIdentifier=discord-bot to actually log your messages as discord-bot[PID] which is even better than python...
    – filbranden
    Apr 24 '18 at 17:11
  • @FilipeBrandenburger As far as I know it is not calling any external programs. At the very least, I know that all the stdout comes from the python process. SyslogIdentifier looks like a decent improvement, thank you.
    – mxbi
    Apr 24 '18 at 18:46

Through your experiment with unbuffer, you have established that the buffering is happening in the command you are running, not in systemd.

So the buffering is happening in the python code you are wrote. To solve that, you need to review the code you wrote or post the code here with a question.

  • What should I look out for within the python code? The things that are getting buffered are simple print statements, for example.
    – mxbi
    Apr 24 '18 at 20:06
  • @mxbi There is already a Q&A on python output buffering. Apr 25 '18 at 13:49
  • Mark's answer is a bit obtuse, but it's correct. At least when run from systemd, Python's buffered output means that it won't write, even if you use systemd's StardardOutput=file:/some/path -- that gets buffered too. All you can do is un-buffer with python's -u switch. Here's some python code, and a unit file. Try swapping the different ExecStarts and commenting out StandardOutput to experiment. It's not the journal buffering, it's Python.
    – mike
    Jul 13 '18 at 12:10
  • fyi, when I run ticktock.py like this: ./ticktock.py > /tmp/ticktock.log then that log is also buffered and not updated until the script ends
    – mike
    Jul 13 '18 at 12:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.