15

I have noticed recently that sometimes tail -f <logfile> will stop updating to the screen.

Doing a Ctrl>-C and restarting the tail works fine, though. And I checked to make sure the logfile isn't being rotated midstream (which can make tail lose its mind).

What would cause this? I'm running RHEL 5.2 x64.

3
  • Does this happen on all logfiles, or just certain ones? What filesystem is the logfile on? What application is writing the logfile? Have you timed the failure to see if it happens either (1) a certain period of time after you started the tail command; or (2) at certain fixed intervals (eg always at :00 :10 :20 :30 :40 and :50 after the hour)?
    – daveadams
    Feb 8, 2011 at 18:42
  • @daveadams - so far as I've noticed, it's only been on two (one on each of two servers): in this case, the dataminer logfile for BSAE (reporting tool) on an HPSA Core server (data center automation) and the server.log for BSAE on the reporting host
    – warren
    Feb 8, 2011 at 19:05
  • 1
    as an aside, tail -F will continue to work even if the file is removes and later replaced by another file entirely.
    – Sirex
    Feb 16, 2011 at 16:30

4 Answers 4

6

Try wrapping your tail command with strace if you have it:

strace -Tt -o /tmp/tail.trace tail -f /var/log/messages

Then just for crazy recursive kicks you can tail the strace output (doesnt' matter if that breaks because its going out to a file):

 tail -f /tmp/tail.trace

Mine looks like:

8:39:00 write(1, "ng SMAC\n", 8)       = 8 <0.000026>
18:39:00 read(3, "", 0)                 = 0 <0.000019>
18:39:00 fstat64(3, {st_mode=S_IFREG|0640, st_size=92990, ...}) = 0 <0.000019>
18:39:00 fstatfs64(3, 84, {f_type="EXT2_SUPER_MAGIC", f_bsize=4096, f_blocks=4807069, f_bfree=1924458, f_bavail=1680271, f_files=1221600, f_ffree=820806, f_fsid={-1331083162, -1313908385}, f_namelen=255, f_frsize=4096}) = 0 <0.000021>
18:39:00 inotify_init()                 = 4 <0.000033>
18:39:00 inotify_add_watch(4, "/var/log/messages", IN_MODIFY|IN_ATTRIB|IN_DELETE_SELF|IN_MOVE_SELF) = 1 <0.000041>
18:39:00 fstat64(3, {st_mode=S_IFREG|0640, st_size=92990, ...}) = 0 <0.000019>
18:39:00 read(4,

The -t switches on the time and -T switches on time spent in calls.

Hit return 4 or 5 times to make a bit of vertical space, then wait for it to stop tailing. Hopefully there will be some clues in the output.

11

Try using:

tail --follow=name <logfile>

And see if that works better. You don't have to worry about it being rotated out from under you.


Any pattern to it stopping? Certain length of time? Certain time of day?

7
  • the file is not being rotated-out from underneath tail - it's just periodically (sometime between 2 and 20 hours) stopping to follow anymore.. wish there was more of a pattern :-\
    – warren
    Feb 8, 2011 at 18:29
  • 1
    Ah… possibly not your issue, but ensure you don't accidentally leave a screen window open in copy mode with tail -f running. Copy mode will make command output block eventually (when the buffer fills up).
    – MikeyB
    Feb 9, 2011 at 1:37
  • 1
    Excellent answer. Of course my files were getting rotated out at the top of each hour!
    – alex
    Jul 11, 2019 at 8:09
  • 1
    What does --follow=name do?
    – Geoffrey
    Jan 20, 2022 at 7:42
  • 1
    as per the man page: "Use --follow=name […]. That causes tail to track the named file in a way that accommodates renaming, removal and creation."
    – MikeyB
    Jan 21, 2022 at 21:50
4

Given that both problematic logfiles are written by different components of the same application, I wonder if it's not some part of the logging code for that application that's causing the problem. I propose two tests to get a better idea of what's going on:

  • Note the inode of the logfile (ls -i logfile) prior to starting the tail, and once the tail fails, check it again. If the inode has changed, then the logger is rewriting the entire logfile, which would break tail's connection.

  • Note the last line before tail stops working, and then visit the file and find the first log entry after that line. Do this 3-5 times if possible. If it's a problem with the program doing the logging, the part of the program that wrote the log entry immediately after you see tail break is most likely responsible. If that log entry is always the same, or if it comes from the same component of the program, you may have enough data to submit a problem report to the application vendor.

Good luck.

3

Having the same problem here.

Problem was, that the file i'm watching was mounted from a different machine. The change notify was not propagated through the mount.

Solution was to use tail on the original machine.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .