1

I've got a web server which has files uploaded to it. There is a script which assigns them numeric IDs and stores them in a corresponding subdirectory.

I've now got 32000 of these uploads and that's too many for the server to handle in one directory. The script I'm using does have a way to "namespace" uploads so that ID 12345 - instead of sitting in /files/12345 - would sit in /files/namespaced/000/012/345.

The code can deal with this just fine, but I now have 32000 subdirectories in the wrong naming style. What's the easiest way to go through my existing files and put them in the right place?

0

2 Answers 2

1

This assumes that the directory names don't have any odd characters such as spaces or newlines in their names and that there are no filenames that start with a dot.

Untested.

find /files -type d | while read -r dir
do
    base="000000${dir##*/}"
    new="/files/namespaced/${base: -9:3}/${base: -6:3}/${base: -3}"
    mkdir -p "$new"
    mv "$dir/*" "$new"
    rmdir "$dir"
done

Another way to obtain the new directory name:

new=$(echo "$dir" | sed 's|...$|/&|;s|\(.*\)/\(.*\)/\(...\)$|\1/namespaced/000000\2/\3|; s|\(.*\)/.*\(...\)\(...\)/\(...\)$|\1/\2/\3/\4|' "$dir")
2
  • I changed to mv "$dir" "$new" and then rmdir was redundant, but looks good
    – Gareth
    Jun 14, 2010 at 14:07
  • Oh, and because of that I had to tweak the folder renaming, but that was simple enough
    – Gareth
    Jun 14, 2010 at 14:11
0

I did it a more manual way. the files were at the base level and I created all the folders and manually copied 1000 jpg files at a time in each folder. A part of the bash script was

mkdir 1-1000 1001-1999 2000-2999 3000-3999
ls *.jpg | head -1000 | while read line; do mv "$line" "1-1000"; done 

I would just manually repeat the command with another folder until I'm done I also would just rename the files to a sha1 string if the file order is not important.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .