Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I had a user with more that 100,000 files in a single directory. The machine locks up and become unusable for a long time whilst the Explorer fills up memory and the machine can crash. Is there a simple way of dumping the directory and contents? We tried using the command line deltree and this did not seam much better.

DaveF

Extra -

We have had this problem several times with the same user (actually client). A program creates 100s of thousands of temp files during its run. If the program works correctly it does not matter as they get removed slowing as it is finished with this. But every now and again it dies leaving the mess of files. We have tried using rmdir and del etc methods and they are faster than using explorer but we generally end up with the same problems. The partition get damaged, there is an unexplained locked file, some file has a permission problem, which stops the delete from working. Normally these problem can be fixed with a reboot but in this case as this is a critical system we can't in production time. On Unix you can just do a rm -rf etc and this is quick and there is generally no real problem with it. All the command line tools we have tried don't seam to work reliably.

BTW - the application is being changed but this will not go into production for sometime and I though this problem was probably interesting to others.

Update - because we have MKS Nutcracker on the system (without the command line utilities) we got the programmer to knock together a simple rm. This is several orders faster than del/deltree etc. I find it strange that there is no such simple app readily available as a standard windows system admin tool.

share|improve this question

15 Answers 15

up vote 11 down vote accepted

You could use rmdir:

rmdir /s /q FOLDERNAME

/s means delete recursively, /q is the "quiet" mode.

share|improve this answer
    
We did try this - I will add more to the question. –  David Allan Finch May 27 '09 at 10:15
    
this should have worked for you. Did it work ? –  s_ruchit May 27 '09 at 10:23
    
yes mostly - I have adding what problems we have seen above in the question. thanks for your comment. –  David Allan Finch May 27 '09 at 10:27
    
I am going to accept this answer. But there is a better solution which is to use a simpler unix style rm but as this is not aviable to most user, this is the best choice for 99% of users. There should be a unix style rm implemented for windows which does nothing else but recursively deletes. BTW our programmers MKS remove_tree (see above) thrashed rmdir so the problem is not Windows but the built-in apps. –  David Allan Finch Jun 1 '09 at 9:51

I use to have this problem with servers I was administering. Skip explorer altogether and use the command prompt. Navigate to the folder and "del ."

This way you avoid the overhead of the GUI (explorer is trash) and Recycling bin.

share|improve this answer
    
We did try this - I will add more to the question. –  David Allan Finch May 27 '09 at 10:16

How about moving that temp folder to a different drive/partition? Instead of deleting all the garbage, you could format the drive (either GUI or command line). With quick format, this should be reasonably fast.

share|improve this answer
1  
We tried this and sometimes it would not allow this. The client now runs the program in a different directory every time. –  David Allan Finch May 27 '09 at 12:44
    
Create a hardlink between the path & drive, ie: the hard link "c:\my silly app\where the files stored" points the drive X: in a startup batch –  Nime Cloud Nov 22 '11 at 12:15

Unless you need this for legacy 16-bit applications, try to increase the performance by disabling short file name generation. This can have a significant impact on directory operations with large number of files.

Regedt32 select:
"HKLM\system\CurrentControlSet\Control\FileSystem\NtfsDisable8dot3NameC
reation"=1

Then use:

RMDIR /S /Q [drive:]path

to remove the directory with all its files.

share|improve this answer
    
That is interesting to know. –  David Allan Finch Oct 22 '09 at 14:18

This is a combination of the previous posts

open a command window:
File -> Run -> cmd

Remove the directory
rmdir /s /q FOLDERNAME

OR

Remove a pattern of files
cd \....\directory
del *.[something]

As you've noted in your question Windows Explorer can really slow things down.

After seeing your update, I have one of my own
Check out this List of file removers

share|improve this answer
    
I had noticed this with only several 100s of files in the past it is a shame that explorer can be made to do it as quick as the command line apps - they we could get the client to do it themselves. Thanks for your answer. –  David Allan Finch May 27 '09 at 10:39

I have nothing better than del and rm, but even if they take a very long time, it should not result in partition damage. The locked files may be because the program that created them died unexpectedly.

Sometimes, even though the files cannot be deleted, they can be moved to another temporary subdirectory. I just create a _todel subdirectory and move them there. It usually works better if I move the entire directory one level up, rather than the files themselves.

Other than that, the only way to make the deletes go faster is to defrag the hard disk. A temporary workaround may be to move this directory to a samba share, or use an NTFS junction to a portable USB hard disk. (A small partition would be better and faster than using the entire hard disk). You can then just eject the hard disk, and do a quick format on another PC.

share|improve this answer
    
That is what I think - IE the damaged files but why are they not closed by the NT Kernel? Speed is a problem with the program so using Samba or a USB hard disk is a non starter. But the client has been suggested to create a separate partition for each run and to reformat. Not sure if they have tried it yet. - Thanks –  David Allan Finch May 27 '09 at 12:52

A damaged partition from deleting files? That's a major problem with the disk, regardless of the number of files being deleted.

I'd try the command line delete and delete as many as you can until you come across one that is locked. Then use Process Explorer to find what process has that file open. Kill that process (if you can), and then do the delete again. Rinse, repeat :)

share|improve this answer
    
Would like to try this but we don't have direct access to the machine and the application has died. Thanks anyway. –  David Allan Finch Jun 1 '09 at 9:42

Reboot using a 'Live' Linux CD, the rm -rf

share|improve this answer
    
We can't just reboot this server as it is critical for the client - but thanks for the answer. –  David Allan Finch Jun 1 '09 at 9:38

Kinda hacky, but you can create a script that will chunk delete the files, i don't know the format, but assuming it's files that start with the letters a-z, you can do

deltree /y a*
deltree /y b*
...
deltree /y z*

change deltree with your removal method of choice. probably be slower, but less error prone?

sidenote: you can also try installing cygwin to get commands like rm which probably would continue deleting files even after it encounters an error with one file.

share|improve this answer
    
This was tried but it it did not really solve the problem - thanks anyway. –  David Allan Finch Jun 1 '09 at 9:38
    
We also have MKS Nutcracker installed but for security reasons the command line utilities where removed - not choice. But what we have done is written out own rm and this does work. –  David Allan Finch Jun 1 '09 at 9:40

The various methods above all work. If you have cygwin loaded on a thumb drive, you can just plug it in to your local USB, fire up your cygwin shell and run the RM command as referenced above. This is a handy way to keep unix flexibility close at hand.

share|improve this answer
    
This is a reasonable solution when you manage the system. –  David Allan Finch Jul 22 '09 at 13:58

You could also boot into the recovery console, or something along those lines. That should completely ignore any strange issues with the OS.

share|improve this answer
    
Sorry we can't just reboot the server but thanks for the answer. –  David Allan Finch Jun 1 '09 at 9:40

Mount the folder from a linux machine and do the recursive delete from there.

# mount -t cifs //server/share /mnt/tobedeleted -o username=yourshareusername

# rm /mnt/tobedeleted/* -R

You could even use a cron job to delete the files at a regular interval.

share|improve this answer
    
have you tried this? –  David Allan Finch Dec 20 '12 at 13:38

If none of the other suggestions are working, have you considered speeding up your I/O?

Buy better controllers, faster disks..

Or I wonder, if the files are temporary (as you imply) and sizes permit, you could add extra RAM to the box (cheap) and set up a RAM disk?

Best case, performance wouldn't be degraded when deleting large volumes of files. Worst case, a reboot would clear the partition.

share|improve this answer
    
Love to but I don't think the client will shell out as they are expecting a new version of the program any - thanks for the answer. –  David Allan Finch May 27 '09 at 10:37
    
Server RAM in the UK is currently about GBP 10 per gigabyte. –  tomfanning May 27 '09 at 11:28
    
True but the customer pay X for the machine (from a quote) and sees no reason to spend Y more. –  David Allan Finch Jun 1 '09 at 9:41
    
Because the specified server doesn't do the job well enough? –  tomfanning Jun 1 '09 at 13:17
    
It is good enough for the job. The problem is the program has a bug that is in the process of being fixed. They are not going to spend any more money. –  David Allan Finch Jun 2 '09 at 9:29

Try cut-and-paste it into the recycle bin, it should bypass trying to move every single file. Not sure what it will do to emptying the bin though because then it probably will go over every file.

share|improve this answer
    
I think the client tried this and it was not much better than using Explorer normally. Thanks –  David Allan Finch May 27 '09 at 12:49

Does deleting with SHIFT+DEL make any difference as to the speed?

share|improve this answer
    
When I tried it last it was much the same. –  David Allan Finch Jul 22 '09 at 13:57
    
In fact I just have tried it again - with an old installation and I am still waiting. I will probably give up and used the MKS rm. –  David Allan Finch Jul 23 '09 at 9:18

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.