0

I launch on a Gnu/Linux server an application myApp:

./myApp &>myApp.log &

Of course, the log file will soon become huge.

I would like to use sed to reduce it (I'm aware of logrotate but my application is still a draft).

But if I use such a command

sed -i "1,200d" myApp.lo

file is indeed truncated but after that, no more data are written into the file by the application.

Why?

3 Answers 3

6

When a file is opened, the system assigns a file descriptor to it. While the file is open, it is accessed via this FD.

When you run the sed command it truncates the file by copying it to a new file. You can see this because the inode number changes

ls -i log
131122 log
sed -i "1,200d" log
ls -i log
131090 log

What's happening is that the file is being written to, you truncate it by copying the contents to another file however the original file is held open by myApp and is still being written to by myApp, it's just marked ready for deletion when myApp finishes. You can see this with lsof.

lsof | grep myapp.log
myapp    11334  iain    1w  REG  253,2   54 131122 /home/iain/myapp.log
sed -i "1,200d" myapp.log
lsof | grep myapp.log
myapp    11334  iain    1w  REG  253,2  416 131122 /home/iain/myapp.log (deleted)

If you can, you should modify your app so that it will accept a signal which tells it to close and then reopen the log file, this will allow it to play nice with something like logrotate.

If you cant modify your app to handle signals then you can use a fifo. I wrote an explanation of how to do this here some time ago.

4
  • @lain -> I can't find the topic you speak about. (search fifo user:lain). May you post the url here? Thank you.
    – Sylva1n
    Jun 13, 2014 at 7:46
  • @Sylva1n I already provided a link under the relevant text.
    – user9517
    Jun 13, 2014 at 7:48
  • @lain sorry...link color is subtil
    – Sylva1n
    Jun 13, 2014 at 8:20
  • @Sylva1n it is a well known bug meta.serverfault.com/questions/6298/…
    – user9517
    Jun 13, 2014 at 8:21
3

Your sed command will create a new file and write the processed output to that file. Then it will rename the new file to replace the old file.

The process will still be writing to the same file you redirected the output to at first. The file is now deleted, but that doesn't prevent a process which has the file open from writing to it.

You should start preparing for using logrotate. It won't address all the issues, but one issue it can do is to keep multiple logfiles around. It isn't a showstopper that a process keeps writing to the file for a little while after it has been rotated.

However a full solution require a bit of coding on your part. Your application need to open the logfile for writing either periodically or when instructed to rotate. That means the application also need to know the filename of the logfile.

3

When you run the ./myApp &>myApp.log command output is redirected to a new file with the filename myApp.log.

When you run your sed -i "1,200d" myApp.log command a new file is created with the name myApp.log.

The problem here is that the original command is still writing to the file that was original created, but which has been unlinked and no longer has a name in the filesystem. That file still remains, though, as it's still open.

The result of this is that not only do you not see the output anymore after running the sed command, if your goal was to free space that does not happen either until the original file is closed.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .