1

In my Python script I needed a fast/efficient way to set a max filesize on a file I'm constantly writing to. Rather than bring the whole thing into py's RAM, I ran this shell command:

sed -i '1d' file.csv

I monitor the filesize periodically and run the command as needed. Problem is that now if I tail -f file.csv, tail stops tailing the file as soon as sed removes a line from it. Any solution?

2
  • 2
    How about tail -F? Sep 8, 2018 at 16:30
  • Do you find any of the suggestions you received helpful? Please provide an update as to how you are faring.
    – simlev
    Sep 15, 2018 at 21:19

2 Answers 2

1

The basic problem is sed creates a new file. This is shown by the extract below:

$ strace -fe trace=file,read,write,close sed -i '1d'  /tmp/x.csv 
....
openat(AT_FDCWD, "/tmp/x.csv", O_RDONLY) = 3
...
openat(AT_FDCWD, "/tmp/sed1nFxY1", O_RDWR|O_CREAT|O_EXCL, 0600) = 4
write(5, NULL, 0)                       = 0
read(3, "1,1,1,1\n2,2,2,2\n3,3,3,3\n", 4096) = 24
write(4, "2,2,2,2\n", 8)                = 8
write(4, "3,3,3,3\n", 8)                = 8
read(3, "", 4096)                       = 0
close(3)                                = 0
close(4)                                = 0
rename("/tmp/sed1nFxY1", "/tmp/x.csv")  = 0
close(1)                                = 0
close(2)                                = 0

There isn't, as far as I know, an easy way to truncate the start of a file.

I'd suggest getting your python application to listen on a unix/tcp socket, and once accepted put that stream into the logging framework and then use nc ... | tail to watch output.

1
  • This is a good explanation, but not really a quick solution...
    – simlev
    Sep 13, 2018 at 15:19
1

Michael Hampton suggests the obvious tail -F, but I personally mostly use less nowadays:

less --follow-name +F file.csv

There being no shorthand alternative to --follow-name, I have to use an alias:

alias less='less --follow-name -XFR'

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.