0

I have a long running python program that logs to a file.

I start program and let it run in the background

python my-process.py >> /var/log/my_log.log  2>&1 &

The free memory slowly declines until logrotate rotates the log file. Memory is freed up, and the process repeats itself.

Does the free memory decline because when I append (>>) to the log file, it keeps the entire log file in memory?

I imagine adding a maxsize property to my logrotate config will keep memory freed up, but I'm also wondering if there is a better way to handle this.

Improve this question
3
  • Are you sure it's not the python program that's eating your memory?
    – glenn jackman
    Jul 12, 2019 at 10:34
  • I've done a lot of investigating of the python program and haven't found a memory leak (though doesn't mean I haven't missed something). Since the memory is almost entirely freed up when my_log.log is rotated, I'm leaning toward it being an issue of the logfile being kept in memory
    – tkwargs
    Jul 12, 2019 at 11:21
  • Did you use iostat or similar tools to capture some data? This seems like a simple filesystem I/O issue.
    – Tux_DEV_NULL
    Jul 12, 2019 at 13:32

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .