Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a file server, which is in-charge of pulling a folder tree from multiple workstations on a daily basis. My current method for this is by using rsync, (which works pretty well provided directory names and/or files remain the same) however, when files are renamed or moved about within subdir1, rsync will copy them over to the server, creating duplicates.

I have to manually find and delete extraneous files/folders that had been left on the server during previous syncs. Note that I cannot use rsync's --delete flag because any sync from a workstation will then mirror that particular folder tree, instead of merging them to the server.

Visual diagram:

Server:         Workstation1        Workstation2        Workstation(n)
Folder*         Folder*             Folder*             Folder*
-subdir1        -subdir1            -subdir1            -subdir(n)
  -file1          -file1              -file2              -file(n)
  -file2
  -file(n)
  • Is there a simple script (preferably in bash, nothing fancy) that can accomplish the deletion of the extraneous files/folders in the event a file is renamed or moved to a different subdir?
  • Is there a different program, much like rsync that can accomplish this task autonomously and in a much simpler manner? I have looked at unison, but I did not like the fact that it keeps a local database for the syncing info.
  • Any tips at all as to how I am supposed to tackle this?

Thank you in advanced for your help.

EDIT: I have tried unison just recently and I can safely say it is out of the question now. unison is a bi-directional synchronization tool and from my testing, it mirrors the files existing on the server to all workstations. - This is unwanted.

preferably, i would want files/folders to stay within their respective workstations and just merge to the server. AKA uni-directional sync; but with renames/moves propagated to the server.

I might have to look into Git/Mercurial/Bazaar as mentioned by kyle, but still unsure if they are fit for the job.

share|improve this question
    
If I were to use Git/Mercurial/Bazaar for this, can it be done so that the commit and whatnot be done via scripts/automated? - forgive the question, I have no experience in using any of those utilities. –  Rey Leonard Amorato Mar 22 '12 at 18:59
    
Could you clarify what you mean by "Note that I cannot use rsync's --delete flag because any sync from a workstation will then mirror that particular folder tree, instead of merging them to the server"? –  amcnabb Mar 22 '12 at 19:04
    
@amcnabb if you look at the diagram, files from workstation1,2,etc are synced to the server. Now, if I were to use rsync --delete workstation1/Folder/ server:/Folder the process would remove file2 from workstation2, and file(n) from workstation(n), etc... Leaving only workstation1's file. –  Rey Leonard Amorato Mar 22 '12 at 19:11
    
Then how can you tell the difference between Workstation1 moving the file and Workstation1 never having the file to begin with? If you need such clever logic, then Unison is a good solution. –  amcnabb Mar 22 '12 at 19:42
    
If workstation1 did not have the file to begin with, it would not exist on the server. File1 is there because workstation1 had it before, then let's say he/she moved it to subdirX, or even renamed subdir1 to subdirX, the server would have copies of both /subdir1/file1 and /subdirX/file1 –  Rey Leonard Amorato Mar 22 '12 at 19:52

1 Answer 1

I think Unison or even Git/Mercurial/Bazaar may be a better choice for this.

A bash script would be extremely difficult to do since the rename of a file would be impossible to differentiate from the creation of a new file, which is what you're running into with rsync.

If it were me I would try to use Git or Mercurial. That would probably require some additional user training, but it would really be the best option. Also, it gives you the benefit of version control so you can revert mistakes.

share|improve this answer
    
I have long wanted to try other solutions as you mentioned, unison/git or other subversion utils, but the sheer complexity they have compared to rsync is what's stopping me. Also, if I were to use git, will the merged folder on the server be reflected on the individual workstations too? -cause this cannot happen. Files belonging to the workstations must stay in their respective workstations. –  Rey Leonard Amorato Mar 22 '12 at 18:29
    
+1 for Mercurial in this situation. It's simple enough that your users should be able to use it with a small amount of training and way more appropriate for the task at hand. –  Sean O'Leary Mar 22 '12 at 18:29
    
@ReyLeonardAmorato Git is much more complex than Mercurial, but also really good for customizing. Mercurial isn't too bad as far as complexity so if you can do maybe an hour training session with users you will probably be ok. If you did do git/hg it would be hard if not impossible to merge the files properly without the changes being reflected on the other workstation's machine. Is there a particular reason the workstation can't be allowed to see other work station's changes? –  Kyle Mar 22 '12 at 20:05
    
Hm. So, what you are saying is that the users have to go to the terminal and manually commit their changes to the repository? Can't it be done automatically? Say after office hours or so.. These people have no interest whatsoever in the technical stuff. –  Rey Leonard Amorato Mar 22 '12 at 20:10
1  
Maybe take a look at this: github.com/ianb/git-sync it allows you to use Git basically as rsync, but it should keep track of all file changes. Then just hook it up as a cronjob or scheduled task. The only thing you may have to worry about is file conflicts on the server. –  Kyle Mar 22 '12 at 20:22

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.