Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Sometimes I have two trees that used to have the same content, but have grown out of sync (because I moved disks around or whatever). A good example is a tree where I mirror upstream packages from Fedora.

I want to merge those two trees again by moving all of the files from tree1 into tree2.

Usually I do this with:

rsync -arv tree1/* tree2

Then delete tree1.

However, this takes an awful lot of time and disk space, and it would be much easier to be able to do:

mv -r tree1/* tree2

In other words, a recursive move. It would be faster because first of all it would not even copy, just move the inodes, and second I wouldn't need a delete at the end.

Does this exist ?

As a test case, consider the following sequence of commands:

$ mkdir -p a/b
$ touch a/b/c1
$ rsync -arv a/ a2
sending incremental file list
created directory
./
b/
b/c1
b/c2

sent 173 bytes  received 57 bytes  460.00 bytes/sec
total size is 0  speedup is 0.00
$ touch a/b/c2

What command would now have the effect of moving a/b/c2 to a2/b/c2 and then deleting the a subtree (since everything in it is already in the destination tree) ?

share|improve this question
    
this is a very good question. where i work ppl have solved the problem using some complex bash and awk scripts –  biosFF Jun 11 '09 at 20:45
    
My preferred inefficient approach is cp -R tree1/* tree2 followed by deletion of tree1. –  romkyns Aug 8 '11 at 13:02

10 Answers 10

Per the mv(1) manpage from gnu's mv:

-u, --update move only when the SOURCE file is newer than the destination file or when the destination file is missing

share|improve this answer
2  
Are you sure this works ? For example; [thomas@ana ~]$ mkdir -p a/b [thomas@ana ~]$ touch a/b/c1 [thomas@ana ~]$ rsync -arv a a2 sending incremental file list created directory a2 a/ a/b/ a/b/c1 sent 125 bytes received 39 bytes 328.00 bytes/sec total size is 0 speedup is 0.00 [thomas@ana ~]$ touch a/b/c2 [thomas@ana ~]$ mv -u a/* a2 mv: cannot move a/b' to a2/b': Directory not empty [thomas@ana ~]$ mv -u -T a a2 mv: cannot move a' to a2': Directory not empty –  Thomas Vander Stichele Jun 12 '09 at 9:42

the proposed mv -uf dir1/* dir2/ move the (sub)directories, not each file. you might try to use find

cd dir1
find . -type d -exec mkdir -p dir2/"{}" \;
find . -type f -exec mv -uf "{}" dir2/"{}" \;

or something similar

share|improve this answer

Does not

mv -uf tree1/* tree2/

work?

share|improve this answer
    
Nope: thomas@ana ~]$ mv -uf a/* a2/ mv: cannot move a/b' to a2/b': Directory not empty –  Thomas Vander Stichele Jun 12 '09 at 9:47
    
mv: cannot move includes' to a subdirectory of itself, ../includes' - so it doesn't merge –  romkyns Aug 8 '11 at 12:53

Midnight Commander (mc) is also nice for this kind of stuff. Tag files with CTRL-t, press F6, and when it asks to overwrite destination files, choose Update if you want to overwrite older files.

share|improve this answer

You could use "cp -l & rm" for in-device moving:

cp -alv --backup=numbered tree1/* tree2 &&
rm -rf tree1/
  • -l of cp to use hard links instead of copying, (this also prevents cross-device operations)
  • --backup=numbered of cp for backing up the existing files in target directory

And be careful about this two issues:

  • use && to prevent your uncopied data to be removed, if you accidentally run it on cross-device targets. (in corss-device case cp exits with a status of "1", at least for GNU coreutils)
  • files starting with "." in tree1, you will lost them if there is any.
share|improve this answer
    
cp is terrible at this sort of thing because it typically destroys hard links / soft links and replaces them with individual files. Similarly it can't be trusted to keep ACLs straight and sometimes gets confused about sparse files. –  chris Jun 28 '13 at 17:57

Javier's answer with find works well, except that it doesn't remove the original directories. Add at the end:

rmdir $(find . -type d  |grep -v ^\.$)
share|improve this answer
1  
Howie, Welcome to ServerFault! Since your answer adds on to someone else's it might be best left as a comment to their answer. Or, you can edit your answer to fulfill the question in it's entirety. (If his question were deleted, yours should stand on it's own.) –  Aaron Copley Jun 28 '13 at 17:44
    
This fails if directory names contain whitespace. Better: find . -type d -not -name '.' -delete (or ! instead of -not) –  Dennis Williamson Jul 8 '13 at 21:54

err

mv dir1/* dir2/

or simply

rsync -arv --remove-source-files  tree1/* tree2

should be enough, you will probably run into trouble at some point when too many entries are in dir1.

find sourcedir -maxdepth 1 -exec echo mv {} targetdir/ \;

should be a nice options

find sourcedir -maxdepth 1 -print0 |xargs -0 -I _ echo mv _ targetdir
find sourcedir -maxdepth1 -exec mv {} targetdir/ +

are both not really necessary because mv just takes 2 options (source target), so you will have to live with the multitude of processes in that case.

share|improve this answer
    
This is not what I'm asking for. Your solution goes only one level deep and doesn't recurse into subdirectories. –  Thomas Vander Stichele Jun 12 '09 at 9:48
    
It doesn't recurse because with the subsequent mv command it's not necessary to recurse any further than one directory. However since you started with rsync I recommend my the --remove-source-files switch which will keep you with the tool you know plus it won't use as much space as the rsync/rm combination. –  Server Horror Jun 12 '09 at 10:02
    
On another note: since I just read your post again regarding the merging you might want to play with -n or -i options from mv –  Server Horror Jun 12 '09 at 10:03
    
I'm pretty sure you're not considering the fact that most of the tree already exists in the destination, so your first mv command will not work. Not sure what -n or -i will help here. Please try the example. –  Thomas Vander Stichele Jun 13 '09 at 7:41
cd /tree1
mv * /tree2

This won't move hidden files or folders, but your original example wouldn't either.

share|improve this answer

I think mv doesn't do what you think it does.

A unix filesystem has 3 components:

  • directory entries
  • inodes
  • blocks

A directory entry points to an inode.

The inode has the metatadata about the file (is it a file, a directory, a named pipe? Who owns it? What are the permissions? What blocks does that inode use?

Blocks are the things that actually contain the contents of the file.

So -- when you "mv" a file, all you're really doing is unlinking the first directory entry and relinking it somewhere else.

snoopy -> inode 333 
woodstock -> inode 333

No data ever gets duplicated / copied. You create the link snoopy, then you create the link woodstock, and then you delete the link snoopy. (things are a bit different with directories because typically you can't make hardlinked directories, but even so the "link" name just changes).

What if you're moving from one filesystem to another? In the old days, mv would just thrown an error and make it explicit that you can't move a file from one filesystem to another. These days, it seems like mv silently copies the data then deletes the original.

In the old days, because you couldn't move data from one file system to another, you got in the habit of using idioms like

tar -cf - . | (cd /new/location && tar -xf -)

then you delete the old data. Part of the reason for using tar was that in the old days, cp would destroy metadata like "this is a symbolic link" and "that's a hard link" and instead you'd just get new copies of that file as regular files. Even still, you need to give "cp" flags to tell it to preserve that sort of structure.

There is no way to avoid "moving" lots of data if it is from one filesystem to another. It doesn't matter if you're using fancy new move or rsync or tar or cpio.

But, if you keep all the data in the same filesystem, this:

mv /filesystem-1/big/directory* /filesystem-1/big2/

that will be extremely fast because it is just changing the directory entries and not actually moving any real data.

There are other issues at play, such as what should you do if there is already a file / directory in the new location as well as the source location?

share|improve this answer

switch to directory you want to move and do

tar cf - * | ( cd /target; tar xfp -)

Quicker than mv...

share|improve this answer
1  
You meant quicker than cp right? –  rkthkr Jun 11 '09 at 14:16
2  
Quicker the mv only if source and target are on different filesystems. –  MikeyB Jun 11 '09 at 18:05
2  
This doesn't move inodes at all. –  Thomas Vander Stichele Jun 12 '09 at 9:50

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.