Using rsync from msysgit for binary files

Question

I have three windows boxes that are connected to a linux box on my network in various ways (one's local to the network, one is a laptop and may or may not be local, and the other is my work desktop). I'm currently making use of git to syncronize most of my stuff between the boxes (using the Linux box as the server) and it works like a champ.

Now, however, I'd also like to add my MP3 collection (12 gigs) to be efficiently synchronized between boxes. I initially thought I could use git for this, but it looks like rsync would be a better choice since I don't really need all the historical stuff. Since all my existing management scripts run under git bash (instead of cygwin), I'd like to be able to use it from git bash (MINGW32 is what it says in the title bar of the window). Can I just download it in Cygwin and copy it over or is there a better way to approach it? The Linux box is running the latest version of Ubuntu Server edition - do I need to do anything to it to enable rsync to work correctly on it?

I realize I'm probably doing this in a slightly harder way than is needed, but I'm also kind of using this as an exercise in improving my commandline skills. Any suggestions?

---

I can use rsync from my cygwin shell, but not from msysgit. I tried copying rsync.exe from cygwin's bin folder over to msysgit's bin folder, but it still isn't finding it. Is there some other step I need to do to install it there? It's strange that I'm getting an error that it can't find the file, instead of giving me an error that one of the file's dependencies is missing.

Answer 1

As of now (9/7/2017) it is extremely easy to manually add rsync support to the Git for Windows environment without even having to resort to running the Git for Windows SDK.

I stumbled across this post and didn't think it could really be that easy given all the alternatives, but it really is. All you need to do is grab the rsync package from the MSYS2 Package repository and extract the rsync.exe file into C:\Program Files\Git\usr\bin. One and done. It works.

If you need help extracting rsync.exe from the rsync package (which is in tar.xz format), you can either use a compression utility like 7zip or do it directly within the Git for Windows bash environment like so:

cd /c/downloaded_location
tar -xvf rsync.pkg.tar.xz usr/bin/rsync.exe --strip-components=2

This gives you the rsync.exe file in the downloaded location (adjust the package name accordingly). In light of Windows permissions issues I won't try to give specific commands on getting the exe into the C:\Program Files\Git\usr\bin folder. You can copy it via an explorer GUI or via an elevated command line utility (and yes, it will require admin level permissions to copy something into the Program Files folders).

If you want to also use Pageant authentication with rsync over ssh, the ssh-pageant binary is already installed in Git for Windows and all you need to do is add a small bit to your ~/.bashrc or ~/.bash_profile file (see ssh-pageant usage section and Git for Windows wiki). I've modified it specifically for use with the Git for Windows environment and enhanced it with some detection for a missing socket file:

# ssh-pageant allows use of the PuTTY authentication agent (Pageant)
SSH_PAGEANT="$(command -v ssh-pageant)"
if [ -S "$SSH_AUTH_SOCK" ]; then
  PAGEANT_SOCK=$SSH_AUTH_SOCK
else
  PAGEANT_SOCK=${TEMP:-/tmp}/.ssh-pageant-$USERNAME
fi
if [ -x "$SSH_PAGEANT" ]; then
  eval $("$SSH_PAGEANT" -qra "$PAGEANT_SOCK")
fi
unset SSH_PAGEANT
unset PAGEANT_SOCK

I can't speak to how other utilities might or might not work by manually installing them this way, but rsync appears to work just fine. Hopefully that repo will stay and continue to be updated--I'd think so because it's hosted at the official MSYS2 site.

Answer 2

I know the question isn't "Using rsync for MSYS", but I think mentioning this is relevant:

"MinGW Shell" (formerly best known as "MSYS (Basic System)") does indeed have rsync. I'm using that shell/system as often as possible since it's much smaller than Cygwin. (I'm probably the only one that cares about file sizes these days.)

"Git for Windows" / "Git Bash" could easily add rsync since it based on MinGW/MSYS and it has been mentioned on the mailing list earlier.

PS! If your MinGW Shell doesn't have rsync, just run "mingw-get install msys-rsync" to add it.

Updated 25th of March 2016:

Git for Windows now uses MSYS2. And MSYS2 uses "pacman", not "mingw-get", to install packages. Unfortunately, "Git for Windows" doesn't include pacman ... There are two options:

1. Install the Git for Windows SDK (which includes pacman).
2. Install Git inside MSYS2.

The last option is my favorite, but it's not streamlined yet so the first option is probably easier. Hopefully, it will be fixed.

PS! I haven't played much with MSYS2 / pacman, but it seems very promising.

Updated 7th of August 2019:

If you are using Git for Windows (Git Bash) it's much better to just grab the rsync package from the MSYS2 Package repository, as is described in the answer above.

Answer 3

I assume by git bash you really mean msysgit, aka "git for windows." The Start Menu option is git bash, which explains the name you used..

First, you are correct in dismissing git as an option here. One of the things git does not do well is handling binary files--and you'd end up with a huge, bloated repository. Additionally, since you won't be handling changes, using the overhead of git strikes me as overkill.

Secondly, it sounds like the question you're actually asking is, "Can I use rsync from msysgit/Windows?" The short answer is, not out of the box. msysgit is really just the Windows command line in terms of form and function. That being said, you have two options: download a third-party tool to enable rsync on your Windows machine (Stack Overflow: Rsync for Windows), or just use cygwin. Really, they have the same outcome--so its up to you to decide which ones work better.

Answer 4

Update January 2021:

Since mysys2 is now using zstd you need to download more than just rsync to get this working. As well as rsync-3.2.3-1-x86_64.pkg you also need to download and copy libzstd-1.4.8-1-x86_64.pkg libxxhash-0.8.0-1-x86_64.pkg and for completeness I also downloaded zstd-1.4.8-1-x86_64.pkg

and 7-zip doesn't do the job of extracting the .zst compressed files wither, so I had to get PeaZip.

Answer 5

For what it's worth, I was having similar problems getting cwRsync to work in msysgit when dropped into the bin path, but after trying lots of different things eventually found that simply updating to the latest version (2.10.2) of Git for Windows fixed the issue.

Answer 6

rsync is a great for unidirectional sync. You can either run rsync under cygwin, or for Windows there is also Deltacopy

If you wanted to have bidirectional sync, then take a look at unison

For a cross platform directory synchronisation tool, there is syncthing which runs on most platforms.