Recursively move thousands of files into subfolders windows

Question

I have a Windows Server with >300,000 images in a single folder.

I am feeling a performance hit on the filesystem and I would like to move them into subfolders as has been suggested by many others.

Most of my files are 10 characters + .jpg - beginning with a 1 (eg: 1234567890.jpg)

I would like to recursively create subfolders based on file name (groups of 3 characters) - and move the files into them. So 1234567890.jpg would be moved from c:\images into c:\Images\1\234\567\890.

How can I do this with a batch file or wscript?

Answer 1

PowerShell is really a good tool for this. I'm not going to write a script for you, but I will point you in the right direction.

You can list everything in the directory using get-childitem and them pass it to a foreach loop. Then, use .substring(0,2) to get the first three letters. Save those three to a variable and create the folder using new-item -Name <foldername> -ItemType folder replacing with the variable holding the first three numbers. It would be a good idea to test for the existence of this folder and only do it if it doesn't exist. Lather rinse repeat adding some logic that your unique situation will likely dictate.

Then, you can just use move-item to move the files into the folders based on pattern matching that looks at the first three letters of the file.

---

I know you wanted someone to write a script for you, but give a man a fish, teach a man to fish...

Answer 2

PowerShell would also be my tool; in fact, I have come up with this one-liner (sort of) to run in c:\images:

dir *.jpg | %{ $dst = $_.basename -replace '(...)(...)(...)$','\$1\$2\$3'; md $dst >$null 2>&1; move-item $_ -Destination $dst }

Schedule yourself some PowerShell time, toy around in the ISE and learn.

---

The expanded equivalent is something like this:

dir *.jpg
    | foreach-object
    { 
        $dst = $_.basename -replace '(...)(...)(...)$','\$1\$2\$3'
        md $dst >$null 2>&1
        move-item $_ -Destination $dst
    }

Or, a bit more verbose and with an example in comments:

# if run from "c:\images\", we can get the image list
$images = dir *.jpg

# process images one by one
foreach ($image in $images)
{
    # suppose $image now holds the file object for "c:\images\1234567890.jpg"

    # get its file name without the extension, keeping just "1234567890"
    $filenamewithoutextension = $image.basename

    # group by 3 from the end, resulting in "1\234\567\890"
    $destinationfolderpath = 
        $filenamewithoutextension -replace '(...)(...)(...)$','\$1\$2\$3'

    # silently make the directory structure for "1\234\567\890"
    md $destinationfolderpath >$null

    # move the image from "c:\images\1234567890.jpg" to the new folder "c:\images\1\234\567\890\"
    move-item $image -Destination $destinationfolderpath

    # the image is now available at "c:\images\1\234\567\890\1234567890.jpg"
}

Answer 3

I actually ended up using wscript - since I haven't played around with PowerShell. It ran surprisingly fast. I assumed 10 character filename (before .ext) starting with a 1

Here's a copy of what I ran:

'On Error Resume Next
Set fso = CreateObject("Scripting.FileSystemObject")
length=10
maindir="C:\inetpub\wwwroot\InventoryImages\"
dodir "C:\inetpub\wwwroot\InventoryImages"


set files=nothing
set folder=nothing
set fso=nothing
wscript.quit


sub dodir(sFolder)
Set folder = fso.GetFolder(sFolder)
Set files = folder.Files
For each file In files
if len(file.name)=length+4 then dofile sfolder,file.name
Next
end sub


sub dofile(dir,filename)
wscript.echo dir & " - " & filename
t_filename=left(filename,10)
t_filename=mid(t_filename,2)
'ASSUMING 1 FIRST DIR
'NOT CHECKING FOR EXISTENCE
t_dir=maindir & "1\"

checkcreate t_dir & left(t_filename,3)
t_dir=t_dir & left(t_filename,3) & "\"
t_filename=mid(t_filename,4)

checkcreate t_dir & left(t_filename,3)
t_dir=t_dir & left(t_filename,3) & "\"
t_filename=mid(t_filename,4)

wscript.echo dir & "\" & filename & "->" & t_dir & filename
fso.movefile dir &"\" & filename,t_dir & filename
end sub


sub checkcreate(dir)
if not fso.FolderExists(dir) then
wscript.echo dir & " does not exist"
fso.CreateFolder(dir) 
end if
end sub