Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Lets say I have a directory ḟoo/ which contains a lot of files in some sort of directory structure. I need to keep some of them, but not all of it.

Is there a way to (in place) delete all of them except (say) 500 newest?

share|improve this question

6 Answers 6

up vote 11 down vote accepted

I do this task regularly, and I use variants of the following. It is a pipeline combining various simple tools: Find all files, prepend the file modification time, sort, remove the file modification time, display all lines except the 500 first, and remove them:

find foo/ -type f | perl -wple 'printf "%12u ", (stat)[9]' | \
    sort -r | cut -c14- | tail -n +501 | \
    while read file; do rm -f -- "$file"; done

A few comments:

  • If you are using “bash”, you ought to use “read -r file”, not just “read file”.

  • Using “perl” to remove the files is faster (and also handles “weird” characters in the file names better than the while-loop does, unless you are using “read -r file”):

    ... | tail -n +501 | perl -wnle 'unlink() or warn "$_: unlink failed: $!\n"'
    
  • Some versions of “tail” do not support the “-n” option, so you must use “tail +501”. A portable way to skip the 500 first lines is

     ... | perl -wnle 'print if $. > 500' | ...
    
  • It won’t work if your file names contain newlines.

  • It is does not require GNU find.

Combining the above gives you:

find foo/ -type f | perl -wple 'printf "%12u ", (stat)[9]' | \
    sort -r | cut -c14- | perl -wnle 'print if $. > 500' | \
    perl -wnle 'unlink() or warn "$_: unlink failed: $!\n"'
share|improve this answer
    
I would be careful with rm -f though. –  Michael Kjörling Nov 17 '11 at 10:34
    
Works like a charm! This should be available as an alias with $path and $count params. Thanks a lot! –  Dalibor Karlović Nov 17 '11 at 11:36

This is how I would do it in Python 3. which should also work for other OSs. After testing this, make sure to uncomment the line that actually removes the files.

import os,os.path
from collections import defaultdict

FILES_TO_KEEP = 500
ROOT_PATH = r'/tmp/'

tree = defaultdict(list)

# create a dictionary containing file names with their date as the key
for root, dirs, files in os.walk(ROOT_PATH):
    for name in files:
        fname = os.path.join(root,name)
        fdate = os.path.getmtime( fname )
        tree[fdate].append(fname)

# sort this dictionary by date
# locate where the newer files (that you want to keep) end
count = 0
inorder = sorted(tree.keys(),reverse=True)
for key in inorder:
    count += len(tree[key])
    if count >= FILES_TO_KEEP:
        last_key = key
        break

# now you know where the newer files end, older files begin within the dict
# act accordingly
for key in inorder:
    if key < last_key:
        for f in tree[key]:
            print("remove ", f)
            # uncomment this next line to actually remove files
            #os.remove(f)
    else:
        for f in tree[key]:
            print("keep    ", f)
share|improve this answer

I don't know about the "500 newest", but with find you can delete stuff that's old than X minutes/days. Example for file and older than 2 days:

find foo/ -mtime +2 -a -type f -exec rm -fv \{\} \;

Test first with:

find foo/ -mtime +2 -a -type f -exec ls -al \{\} \;

Mind the backslashes and the space before "\;". See the find man page for more info.

share|improve this answer
    
The “(say) 500 newest” is the essence here, so I don’t see how this answers the original question. –  Peter John Acklam Nov 17 '11 at 13:05
    
Sorry, wasn't clear to me. –  AndreasM Nov 17 '11 at 13:08

if you could do with keeping files x days/hours old instead of the newest x number, you could do it just with tmpwatch --ctime 7d

share|improve this answer

I think the -mtime and -newer options of find command are useful for you. You can see man find for more info.

share|improve this answer

why not use this simpler code:

$ ls -t1 foo/| xargs -d '\n' rm --
share|improve this answer
1  
How does this remove all files except the 500 most recent files? And how does this handle subdirectories? I think you might have misunderstood the original posting. –  Peter John Acklam Nov 17 '11 at 13:03

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.