1

I've got a web server which has files uploaded to it. There is a script which assigns them numeric IDs and stores them in a corresponding subdirectory.

I've now got 32000 of these uploads and that's too many for the server to handle in one directory. The script I'm using does have a way to "namespace" uploads so that ID 12345 - instead of sitting in /files/12345 - would sit in /files/namespaced/000/012/345.

The code can deal with this just fine, but I now have 32000 subdirectories in the wrong naming style. What's the easiest way to go through my existing files and put them in the right place?

1

This assumes that the directory names don't have any odd characters such as spaces or newlines in their names and that there are no filenames that start with a dot.

Untested.

find /files -type d | while read -r dir
do
    base="000000${dir##*/}"
    new="/files/namespaced/${base: -9:3}/${base: -6:3}/${base: -3}"
    mkdir -p "$new"
    mv "$dir/*" "$new"
    rmdir "$dir"
done

Another way to obtain the new directory name:

new=$(echo "$dir" | sed 's|...$|/&|;s|\(.*\)/\(.*\)/\(...\)$|\1/namespaced/000000\2/\3|; s|\(.*\)/.*\(...\)\(...\)/\(...\)$|\1/\2/\3/\4|' "$dir")
  • I changed to mv "$dir" "$new" and then rmdir was redundant, but looks good – Gareth Jun 14 '10 at 14:07
  • Oh, and because of that I had to tweak the folder renaming, but that was simple enough – Gareth Jun 14 '10 at 14:11
0

I did it a more manual way. the files were at the base level and I created all the folders and manually copied 1000 jpg files at a time in each folder. A part of the bash script was

mkdir 1-1000 1001-1999 2000-2999 3000-3999
ls *.jpg | head -1000 | while read line; do mv "$line" "1-1000"; done 

I would just manually repeat the command with another folder until I'm done I also would just rename the files to a sha1 string if the file order is not important.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.