3

The issue is that any modification to the directory locks up Explorer indefinitely, though Samba access to other directories still works. I've tried moving files locally and over Samba.

Even enumerating the directory to get the list of files locks up the computer indefinitely.

I tried using Python's win32file.FindFilesIterator to iterate the files but that also hangs.

My idea was to move each file to a different directory (in a directory above the directory we're dealing with) based on its timestamp, so that we'd have at most a thousand or so files in each directory... But since I can't even enumerate the files, that's been a non-starter.

If I have to give up and just nuke the directory I'm willing to do that, but a standard delete also hangs indefinitely.

I have set these two parameters to increase speed and they also did not help the issue:

R:\>fsutil behavior query disablelastaccess
disablelastaccess = 1

R:\>fsutil behavior query disable8dot3
disable8dot3 = 1

These are all sequential images that would have run into the 'bug' with 8.3 filenames whereby many similarly named files in one directory can take a long time to compute 8.3 filenames. From what I understand this data is stored in the file system even after disable8dot3 is enabled, so it may still be contributing to the problem.

Any ideas?

8 Answers 8

1

Depending on how the files were added and how badly the folder is fragmented the quickest way might even be to move off the files you want to keep and nuke the partition!

1
  • Yup, I tried a few more programmatic approaches and then moved the root directories outside of the file share so that no one could accidentally access them and enumerate anything, then when we got our new storage system we just moved over the good stuff and nuked the old one from high orbit. :)
    – Beau
    Apr 5, 2011 at 17:48
4

I don't know if it will work, neither if it is practical approach for you, but what about putting the harddisk into a Linux Computer and try it with Linux?

(There are bootable Live CDs for download - you don't even have to physically move the harddisk).

3
  • 1
    Ah, I may give this a try with Knoppix! I hadn't thought of that, thanks!
    – Beau
    Feb 3, 2010 at 18:41
  • I don't think that will help, it is simply too many files for a single folder.
    – JamesRyan
    Feb 4, 2010 at 12:02
  • Did it work? would be interested in the result.
    – StampedeXV
    Feb 6, 2010 at 18:18
1

Two things you should not do:

  1. Place so many files into one folder
  2. Use Windows Explorer to view such a folder, let alone try to do anything with it.

What's happening is that Explorer is accessing each and every one of those files for the information that will, or could, be displayed in the Explorer interface. Additionally, if you have on access antivirus scanning taking place (and if not, why not), each file will be scanned, even if it's only to determine whether or not it is one of the types that should be scanned. All this takes a long time when dealing with so many files.

Short term solution - Use the command line or some form of scripting instead.

Long term solution - Develop a sensible scheme for ensuring you never have more than a few thousand files in any one folder.

1
  • I agree wholeheartedly with your advice. :) I have tried non-Explorer solutions and they suffer from the same problems, however... My Python approach, for example, hangs while enumerating the files. We already have a scheme developed to prevent this from happening in the future but I'm left with a server with this directory sitting on it.
    – Beau
    Feb 5, 2010 at 17:50
1

You could learn a little ahk or php. ahk would be easier to learn and someone on http://webchat.freenode.net/?channels=#ahk will probably be willing to help you get something together.

quick ex. from when I needed to do this...
Loop, C:\Users\PC\Web Sites\Folderwithtomanyfiles\*.htm, , 1  ; Recurse into subfolders.
{
Match :=
Match1 :=
name :=
name1 :=
M :=
File :=
total++
;the names of my files were each 7digits with some extra info I didn't want to share here so... 
RegExMatch(A_LoopFileFullPath, "Folderwithtomanyfiles\\(\d{7}.*?htm)", name)
RegExMatch(A_LoopFileFullPath, "Folderwithtomanyfiles\\(\d{7})", Match)
StringTrimLeft, Ma, Match1, 6
M .= Ma . "\"
StringTrimLeft, Ma, Match1, 5
M .= Ma . "\"
StringTrimLeft, Ma, Match1, 4
M .= Ma . "\"
StringTrimLeft, Ma, Match1, 3
M .= Ma . "\"
StringTrimLeft, Ma, Match1, 2
M .= Ma . "\"
StringTrimLeft, Ma, Match1, 1
M .= Ma
FileCreateDir, C:\Users\PC\Web Sites\fewer file names\%M%
FileAppend, %File%, C:\Users\PC\Web Sites\fewer file names\%M%\%name1%
created++
}
MsgBox Created %created% out of %total% 
}
;example of path ... 
;"C:\Users\PC\Web Sites\fewer file names\3\33\233\5233\35233\435233\0435233.htm
;turns out I should have put a few thousand files in each folder but oh well, hope the example helps. 
0

Try using PowerShell. Using the Move-Item Cmdlet

HTH

2
  • What's wrong with the move command at the command prompt? Sure is a whole lot simpler, easier and faster. Feb 4, 2010 at 3:58
  • Nothing wrong with "move" command, but PowerShell will give you more options.. 1. -WhatIf i.e. don't actually execute the command and show's the result of the command 2. Writing a conditional script check this article technet.microsoft.com/en-us/library/ee176988.aspx Feb 4, 2010 at 14:40
0

There's always cmd.exe and its del command.

0

You will need to reboot for the FSUTIL changes to take effect, just in case you haven't.

The general order of complexity for NTFS functions (Add, Delete, Search) with 8.3 names disabled is O(Log N) so pretty much anything you do is going to take the same length of time once things have gotten to this state.

If you have any AV active on the system then turn it off while you fix this.

Out of curiosity do you get better performance if you copy files out in chunks using something like Robocopy? I suspect that it won't make much of a difference but it should take half the time a delete takes (there's only a search of the directory involved not a search then a delete of the entry). You'll still have the problem of deleting the originals eventually though.

0

use robocopy with the /mov option

http://technet.microsoft.com/en-us/library/cc733145.aspx

For example I had a folder with 1.2 m email files in it, which would hang explorer. The files were named starting with the date so I created folders for each month of the year and then used robocopy filtering on the date in the filenames.

robocopy e:\sent-old\ e:\sentmail\2013-4 201304*.* /mov

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .