1

In my Python script I needed a fast/efficient way to set a max filesize on a file I'm constantly writing to. Rather than bring the whole thing into py's RAM, I ran this shell command:

sed -i '1d' file.csv

I monitor the filesize periodically and run the command as needed. Problem is that now if I tail -f file.csv, tail stops tailing the file as soon as sed removes a line from it. Any solution?

2
  • 2
    How about tail -F? Sep 8, 2018 at 16:30
  • Do you find any of the suggestions you received helpful? Please provide an update as to how you are faring.
    – simlev
    Sep 15, 2018 at 21:19

2 Answers 2

1

The basic problem is sed creates a new file. This is shown by the extract below:

$ strace -fe trace=file,read,write,close sed -i '1d'  /tmp/x.csv 
....
openat(AT_FDCWD, "/tmp/x.csv", O_RDONLY) = 3
...
openat(AT_FDCWD, "/tmp/sed1nFxY1", O_RDWR|O_CREAT|O_EXCL, 0600) = 4
write(5, NULL, 0)                       = 0
read(3, "1,1,1,1\n2,2,2,2\n3,3,3,3\n", 4096) = 24
write(4, "2,2,2,2\n", 8)                = 8
write(4, "3,3,3,3\n", 8)                = 8
read(3, "", 4096)                       = 0
close(3)                                = 0
close(4)                                = 0
rename("/tmp/sed1nFxY1", "/tmp/x.csv")  = 0
close(1)                                = 0
close(2)                                = 0

There isn't, as far as I know, an easy way to truncate the start of a file.

I'd suggest getting your python application to listen on a unix/tcp socket, and once accepted put that stream into the logging framework and then use nc ... | tail to watch output.

1
  • This is a good explanation, but not really a quick solution...
    – simlev
    Sep 13, 2018 at 15:19
1

Michael Hampton suggests the obvious tail -F, but I personally mostly use less nowadays:

less --follow-name +F file.csv

There being no shorthand alternative to --follow-name, I have to use an alias:

alias less='less --follow-name -XFR'

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .