11

I have a directory with a couple hundred thousand files in it.

I want to delete all the files, but rm * -f reports: "argument list too long"

What is the best way to clear out this directory?

1

8 Answers 8

11

In the event you cannot remove the directory, you can always use find.

find . -maxdepth 1 -type f -exec rm -f {} \;

That will delete all files in the current directory, and only the current directory (not subdirectories).

4
  • almost perfect (see my post too)
    – asdmin
    Jul 24, 2009 at 17:52
  • 2
    On linux you can do it even faster by using "+" instead of "\;". That will cause it to remove more files per "rm" in one go.
    – Thomas
    Jul 28, 2009 at 13:07
  • 5
    Even faster: find dir/to/delete -delete (Does not spawn a rm process per file to delete). Jul 28, 2009 at 14:42
  • Morten: He never mentioned wanting to delete the directory itself, eh said onky the files. find dir/to/delete -type f -delete
    – richo
    Dec 9, 2010 at 6:07
21
find . -maxdepth 1 -type f -exec rm -f {} \;

it simply takes too long (one exec of rm per file).

this one is much more efficient:

find . -maxdepth 1 -type f -print0 | xargs -r0 rm -f

as it takes as much filenames as argument to rm as much it's possible, then runs rm with the next load of filenames... it may happen that rm is only called 2 or 3 times.

4
  • 5
    Glad to see someone know how find works... Also consider with new gnu find's the -delete action or -exec command {} + which acts like xargs. See the Actions section of 'man find'. Jul 24, 2009 at 18:21
  • I didn't know about -exec cmd {} +. That looks rather useful. Slightly more generic than -delete :) Jul 24, 2009 at 21:16
  • -delete may be fine, but I personally prefer commands which can be used on variety of machines. Solaris systems I work with are older, so -delete would not work on them
    – asdmin
    Jul 24, 2009 at 21:33
  • 2
    Adding ionice just before rm is a good idea if the computer is used by other people at the same time. Apr 26, 2012 at 15:16
6

Both these will get round the problem. There is an analysis of the respective performance of each technique over here.

find . -name WHATEVER -exec rm -rf {} \;

or

ls WHATEVER | xargs rm -rf

The problem stems from bash expanding "*" with everysingle item in the directory. Both these solutions work through each file in turn instead.

6
  • 2
    The find solution is slower with many files because it invokes the rm command for every single file. The xargs solution is faster but it only works when the filenames do not contain spaces (otherwise you need GNU find and find . -print0 | xargs -0 rm).
    – robcast
    Jul 24, 2009 at 16:13
  • "find . -name WHATEVER -print0 | xargs -0 rm -rf" will be more efficient. find will fork an rm for every file, where as xargs will reduce it to the minimum. You need the -print0 and -0 to cope with files with spaces in. You probably also want to use -depth on find, so it does a depth first search. Jul 24, 2009 at 16:13
  • The ls command will not work, as that too returns "argument list too long"
    – Brent
    Jul 24, 2009 at 16:19
  • 2
    You're doing "ls *", and the "*" expands to the argument list that your shell complains about being too long. Do "ls ." instead (or go up one directory level and do "ls [dirname]"). Jul 24, 2009 at 18:03
  • the second command line (ls WHATEVER | xargs rm -rf) would not work for the same reason as the question did not work: WHATEVER would be substituded by shell to an exceeding length
    – asdmin
    Jul 24, 2009 at 21:31
3

I was able to do this by backing up one level:

cd ..

And running:

rm directory name -rf

And then re-creating the directory.

1
  • 1
    it works only if you don't have to have the dir permanently and all the files and directories shall be recursively deleted. most of the cases in my life, this way would have not worked.
    – asdmin
    Jul 26, 2009 at 8:09
2

All these find invocations are very nice but I seldom remember exactly the nomenclature needed when I'm in a hurry: instead I use ls. As someone mentions, ls . would work but I prefer ls -1 as in:

ls -1 | xargs -n 100 rm -rf

The -n xxx figure is pretty safe to play around with as exceeding the maximum will either be auto-corrected ( if size-max is exceeded; see -s ) or if the args-max for an app is exceeded it will usually be rather obvious.

It should be noted grep is handy to insert in the middle of this chain when you only want to delete a subset of files in a large directory, and don't for whatever reason want to use find.

This answer assumes you are using Gnu core utilities for your ls, xargs & etc.

2
  • Will this work with a directory that is too large for "ls" to work? (same error - arguement list too long)
    – Brent
    Aug 5, 2009 at 13:58
  • Yes Brent: just be sure not to use a filespec when invoking ls. Use just the -1 (dash One) as shown above. And as explained above, use grep after the ls if you need to pair down the file list. If you are getting the "too long" error after making sure you're not using any wildcards with ls, then maybe xargs is complaining. Use 'xargs -n 5' instead of 'xargs -n 100' and you should definitely be safe, if a little slower.
    – rixtertech
    Dec 14, 2009 at 18:03
1

You can use the -exec + option to find which which will try to run rm as few times as possible, which might be faster.

find . -type f -exec rm '{}' +
0

Here's a version for deleting large number of files when the system needs to remain responsive.

It works by issuing work in small batches (100 files by default) and waiting a bit for other jobs to finish.

Worked brilliantly for deleting over half a million files from single directory on ext3. It prints percentage done as a little bonus

noOfFiles=`ls -U | wc -l`
n=0
while (ls -U | tail -n 100 | xargs rm -f); do 
   n=$((n+100));
   sync;
   echo -en "$((n*100/noOfFiles))%, ($n of $noOfFiles)\r";
   sleep 5;
done
0

Solves "argument too long" or "cannot allocate memory" errors

This did the trick on 220,000+ files in session folder....

Advantage: instantly starts removing files

cd path/to/folder
ls -f | xargs rm -f -v

CLICK for screenshot of files being removed - (removed all files in ~ 15min)

-f (after ls) keeps from presorting

-v (after rm) displays each file as being removed

-f (after rm) forces through with no prompt on write-protected files

Tip: Rename folder (ex session to session_old) first to keep additional autogenerated files from being added while you are trying to delete files. You can remake original directory manually if it doesn't automatically as in my case

1
  • What’s new in your answer that was missing from the previous ones? Mar 14, 2017 at 21:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.