move files from one device to another

Question

I am trying to automate a remote backup on our centOS server, I run the current command :

nice -n 10 mv -vfb /media/localbackup/* /media/remotebackup/

however when this is ran I get the following error and no files are moved

mv: inter-device move failed: `/media/localbackup/04-21-13' to `/media/remotebackup/04-21-13'; unable to remove target: Is a directory

I have been told it is better to use cp however this won't delete the files afterwards and I need to remove the files upon successful backup.

Answer 1

I also suggest you using rsync instead of mv, but adding to @Hartmut's answer:

rsync -abmv --remove-source-files /media/localbackup /media/remotebackup/
find /media/localbackup -depth -type d -empty -delete

And 2 additions here:

1. The -m (--prune-empty-dirs) parameter to rsync, which on my version 3.0.7 protocol version 30 says: prune empty directory chains from the file-list but doesn't do that for some reason. Doesn't error either (not the only one to notice it).
2. The find command which will delete the empty directories.

now, the reason against cp && rm is that cp might fail half-way through. The next time you want to backup you will be in a weird situation (partially copied files to name the worst case). Also, the next backup will not know what to do with the previously backed up files which were not deleted. On the other hand rsync covers these aspects just fine.

Also, using a script (as first answer suggests) to do what rsync already does, but worse, is a no-go (sorry Hauke Laging :-) ). No need to reinvent the wheel here.

Answer 2

I find I often want to do some operation on files, while preserving their directory structure. For this reason I defined a bash function filtercmd. This takes a list of files to operate on as stdin, and as parameters takes the operation (e.g. mv) and the destination directory. Once I've done this I can do things like this as a simple one liner.

filtercmd () {
if [ ! -d "$2" ]; then echo "$2" does not appear to be a directory; return 1; fi
while read f; do
    dir="${f%/*}"
    if [ ! "$dir" = "$lastdir" ] 
        then mkdir -p "$2/$dir"; echo "$dir"; lastdir="$dir"; fi
    [ -e "$2/$f" ] || $1 "$f" "$2/$f"; done
}

(cd /media/localbackup && find . | filtercmd "mv -vfb" /media/remotebackup)

This can also be done with rsync. Using rsync's remote protocol can save a lot of bandwidth since the protocol only needs to send changed chunks. However, two advantages of filtercmd over rsync are (1) that you can use your existing mv flags without learning the rsync equivalents and (2) and we can also use filtercmd for more powerful things like:

(z(){ grep . <$1|gzip >$2.gz;};find . -name '*.txt' | filtercmd z ../gz })

Which selectively takes txt files, eliminates empty lines, and gzips them onto the destination directory.

Answer 3

Your command tries to create an already existing directory. You should make this shell code more robust:

tmpdirname=tmpdir
test -d "/media/remotebackup/${tmpdirname}" &&
  rm -rf "/media/remotebackup/${tmpdirname}"
pushd /media/localbackup/
test -d "$tmpdirname" && { echo Abbruch; exit 1; }
for dir in *; do
  test -d "/media/remotebackup/${dir}" && continue
  mv "$dir" "/media/remotebackup/${tmpdirname}" &&
    mv "/media/remotebackup/${tmpdirname}" "/media/remotebackup/${dir}"
done
popd

This will skip directories which already have been successfully moved.

Answer 4

I would use rsync command with the --remove-source-files option.

Something like rsync -abv --remove-source-files /media/localbackup/* /media/remotebackup/

Please test before you apply it to your environment.