5

I want to reduce the log file size to the particular minimum size

I tried the truncate command as below

$ truncate -s 20M log filename

But when I check the truncated file it truncates the recent logs and keeps the old logs

When analyzing further I found the tail and head commands. But the problem is when I use the head command is working and when I use tail command it saves the content in the human not readable format like @@@@@@@@@@@.

$ head -c 10M logfile > newfile (it saves the head of the file as mentioned size)

$ tail -c 10M logfile > newfile(It saves the content which cannot be readable)

Now I want to truncate the large file which removes only starting of the file but not the ending of the file and if possible it should be in one command without creating another file as head and tail Can anyone guide me on this? Thank you!

2
  • The sad part is i leave this question unsolvable and after two month for another task am searching for the same question and found my own question in google without knowing its my own question i thought someone asks same as my requirement but no one answered till now!!!! can anyone help me on this? Commented Jul 27, 2022 at 12:26
  • 1
    This is how I do it: cat logfile | gzip -c > logfile.gz; :> logfile
    – Marcel
    Commented Oct 25, 2022 at 8:22

3 Answers 3

2

Being a fan of simplicity and high maintainability, I made this. I'm open to your opinion.

This command will truncate log file /tmp/myfile, it cuts the beginning and not the end of the file, it does not require you to configure services and install multi-line scripts; just a one-line command you can add to your crontab.

tail -c 1M /tmp/myfile > /tmp/dzqxH4ZMiSQb91uMMMgPhsgmpnc && rm /tmp/myfile && mv /tmp/dzqxH4ZMiSQb91uMMMgPhsgmpnc /tmp/myfile

First command cuts 1MB of the target file into a temp file, second command deletes the target file, and the third one renames the 1MB temp file as your target file.

It has the disadvantage that it will allocate in your file system, temporarily, 1 MB of space. This could be a problem if you want to keep, say, the last 1 GB of your log file, and your storage is too small or too slow.

If that's your case, you could try this:

fallocate -c -o 0 -l 1M file

It will cut off bytes from your file; starting at position 0, it deallocates 1 MB of disk space off your file. I understand it does it in place, instead of copying data into a temporary file. But it takes more complexity; you would need to first calculate the current size of the file to know how much to deallocate, or make a little loop, something like:

while (file larger than 100M) fallocate 1M; 

resulting in a file between 100 MB and 99 MB. But I like simplicity so for now I'm going with the first option.

2
  • rm /tmp/myfile??? That means any process with that file open as its log file will continue writing to the file after it's been deleted. Commented Jul 20, 2023 at 17:19
  • True. Any writes to /tmp/myfile done while truncation is going on, would be lost by the final mv command. The fallocate approach does not have this weakness.
    – Lascou
    Commented Sep 4, 2023 at 20:50
1

The approach: tail -c 10M logfile > newfile should work. Try 'which tail' and 'where tail' to see what is wrong.

I have been using tail -c for many years. Since this is getting a bigger thing for me now, I just now wrote a little tcsh script to do it:

#!/bin/tcsh -f
set echo_style='none' ; setenv LC_ALL C #; set echo verbose

# Purpose: reduce the size of log file FILE to SIZE, if it exceeds
#          twice that amount, by junking the beginning of it.

# Usage: reducelog SIZE FILE
# Clobbers FILE.tmp

# check arguments
set msg="Usage: reducelog SIZE FILE"
if ($#argv != 2) goto err
set size="$1"
set msg="Invalid reduced SIZE"
if (`echo "$size" | grep -c '^[1-9][0-9]*$'` != 1) goto err
set file="$2"
set msg="File not accessible: $file"
if !(-frw "$file") goto err

# check whether we should cut
set msg="Error finding the size of $file"
set oldsize="`ls -l "$file"  | cut -d' ' -f 5`"
if ($status) goto err
if (`echo "$oldsize" | grep -c '^[0-9][0-9]*$'` != 1) goto err
@ maxsize = $size + $size
if ("$oldsize" < "$maxsize") goto end

# cut it down quickly (so that hopefully nothing else interferes)
set msg="Error reducing the size of $file"
tail -c "$size" "$file" >! "$file.tmp"
if ($status) goto err
set msg="Error replacing $file"
mv -f "$file.tmp" "$file"
if ($status) goto err

# check that we did reduce the size
set msg="Error finding the new size of $file"
set newsize="`ls -l "$file"  | cut -s -d' ' -f 5`"
if ($status) goto err
if (`echo "$newsize" | grep -c '^[0-9][0-9]*$'` != 1) goto err
set msg="Failed to reduce the file size"
if !("$newsize" < "$oldsize") goto err

# done
goto end

# error
err:
echo "*** $msg\!"
exit 1

# end
end:

The reason I am still googling this is that I am somewhat concerned about the (remote) possibility that some process writes to the log file just as I am moving it into position. But I guess there is simply no other way to find on the web.

Note: in the script above, SIZE is in bytes, e.g. 100000 for 100 kB.

2
  • If the process writing to the log file keeps the file open, then deleting the original file and replacing it with a new file will not accomplish anything. The open file descriptor will still be writing to the now-deleted file and the file created by the tail command will never be touched.
    – doneal24
    Commented Nov 4, 2022 at 20:03
  • @doneal24: Yes, if the process writing to the log file keeps the file open, the above solution does not work by itself. But read the original question: the poster wanted to use tail, (but got some other problem), so this obviously did not apply to him (or me, so far). But I found that fetchmail (in demon mode) does do this. So I had to make an adjustment to my script in this case to touch $HOME/.fetchmailrc immediately after moving the new log into position. This causes fetchmail to restart and reopen the log. Thanks for reminding me to add that here.
    – LeoninT
    Commented Nov 5, 2022 at 21:40
0

Configure logrotate to manage your log files.

What you most probably want is the copytruncate option.

1
  • 1
    I don't want to rotate the log file. Just keep the file and want to truncate the file for the last N lines only. When i try to truncate it deletes the recent logs and keep the old logs but i want vice versa. Can you help me on this? Commented Jul 27, 2022 at 15:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .