I have a simple user systemd service configured like so:

[Unit]
Description=Bot
AssertPathExists=/home/mikel/discord-bot/

[Service]
WorkingDirectory=/home/mikel/discord-bot/
ExecStart=/home/mikel/anaconda3/bin/python -u bot.py parameters.json
Restart=always
RestartSec=3

[Install]
WantedBy=default.target

Essentially, all the service does is run a python script.

However, when I look at the journalctl for the service, I see that instead of logs appearing as things happen in the script, it is all flushed at once when the service is stopped (with the same timestamp for every line).

This suggests that some buffer is occuring somewhere that is preventing it from being written to journalctl immediately. However, even when I turn off Python's internal buffer with -u, the problem still occurs.

One solution I have found is to wrap it in the unbuffer command from the expect package, like so:

ExecStart=/usr/bin/unbuffer /home/mikel/anaconda3/bin/python -u bot.py parameters.json

However, while this works, it means that journalctl shows "unbuffer[PID]" instead of "python[PID]", and I want a solution that doesn't involve installing external packages for what seems like a simple task.

Is there some other way to remove these buffers so that the output is flushed to journalctl as it comes from python?

  • 1
    Can you give more details on what bot.py is doing? Is it calling external programs? That could explain why the output is unbuffered on those and why unbuffer would fix it... – Filipe Brandenburger Apr 24 at 17:09
  • 2
    Maybe not exactly what you want... But look at SyslogIdentifier= config in systemd. You could use something like SyslogIdentifier=discord-bot to actually log your messages as discord-bot[PID] which is even better than python... – Filipe Brandenburger Apr 24 at 17:11
  • @FilipeBrandenburger As far as I know it is not calling any external programs. At the very least, I know that all the stdout comes from the python process. SyslogIdentifier looks like a decent improvement, thank you. – mxbi Apr 24 at 18:46

Through your experiment with unbuffer, you have established that the buffering is happening in the command you are running, not in systemd.

So the buffering is happening in the python code you are wrote. To solve that, you need to review the code you wrote or post the code here with a question.

  • What should I look out for within the python code? The things that are getting buffered are simple print statements, for example. – mxbi Apr 24 at 20:06
  • @mxbi There is already a Q&A on python output buffering. – Mark Stosberg Apr 25 at 13:49
  • Mark's answer is a bit obtuse, but it's correct. At least when run from systemd, Python's buffered output means that it won't write, even if you use systemd's StardardOutput=file:/some/path -- that gets buffered too. All you can do is un-buffer with python's -u switch. Here's some python code, and a unit file. Try swapping the different ExecStarts and commenting out StandardOutput to experiment. It's not the journal buffering, it's Python. – mike Jul 13 at 12:10
  • fyi, when I run ticktock.py like this: ./ticktock.py > /tmp/ticktock.log then that log is also buffered and not updated until the script ends – mike Jul 13 at 12:17

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.