32

I have three windows boxes that are connected to a linux box on my network in various ways (one's local to the network, one is a laptop and may or may not be local, and the other is my work desktop). I'm currently making use of git to syncronize most of my stuff between the boxes (using the Linux box as the server) and it works like a champ.

Now, however, I'd also like to add my MP3 collection (12 gigs) to be efficiently synchronized between boxes. I initially thought I could use git for this, but it looks like rsync would be a better choice since I don't really need all the historical stuff. Since all my existing management scripts run under git bash (instead of cygwin), I'd like to be able to use it from git bash (MINGW32 is what it says in the title bar of the window). Can I just download it in Cygwin and copy it over or is there a better way to approach it? The Linux box is running the latest version of Ubuntu Server edition - do I need to do anything to it to enable rsync to work correctly on it?

I realize I'm probably doing this in a slightly harder way than is needed, but I'm also kind of using this as an exercise in improving my commandline skills. Any suggestions?


I can use rsync from my cygwin shell, but not from msysgit. I tried copying rsync.exe from cygwin's bin folder over to msysgit's bin folder, but it still isn't finding it. Is there some other step I need to do to install it there? It's strange that I'm getting an error that it can't find the file, instead of giving me an error that one of the file's dependencies is missing.

6 Answers 6

40

My original instructions to get rsync working with Git for Windows were quite a bit outdated, so I've revamped this answer for mid-2023 to cover MSYS2's switch to zstd and the additional dependencies required for rsync. The primary issue is that MSYS2 now compresses most packages (including zstd, rsync, and dependencies) using zstd, but zstd isn't included in the base environment. Because of this we can't get zstd from the MSYS2 repository--but we can get it directly from the zstd Github releases where it is a zip archive rather than zst. The basic idea for getting rsync working that I originally came across in this post is still the same, but due to the above mentioned issues it requires a bit more work. The whole process is still easily doable once we know everything required.

Below are commands to:
  • download the zstd binary to use in extracting further packages
  • download, extract, and place the rsync binary and it's dependencies in the correct bin folder

These steps have been tested on a clean install of Windows 11 with only the latest Git for Windows installed (should work for Windows 10 as well). No other dependencies should be required to complete the steps.

As always, examine what you paste into a terminal and use at your own risk

Step 1: zstd requirement

If you have an environment that doesn't already support extracting archives compressed with zstd (such as with Git for Windows), the easiest way is probably to download the latest zstd Windows binary directly from the ztsd release page: https://github.com/facebook/zstd/releases. If you already have zstd available and in your sourced path then skip to step 2 (and you may need to adjust the zstd binary location in the script).

Copy the below lines into a Git Bash terminal after adjusting below variables for latest source locations, versions, and your preferred download location:

working="/c/temp-downloads"
zstd_url="https://github.com/facebook/zstd/releases/download/v1.5.5/zstd-v1.5.5-win64.zip"
mkdir -p "$working"
curl -L -o "$working/zstd.zip" "$zstd_url"
unzip -j "$working/zstd.zip" "*/zstd.exe" -d "$working"

Step 2: Downloading compatible rsync and dependency binaries, extracting, and placing in proper location

Copy the below lines into an admin Git Bash terminal after adjusting below variables for latest source locations, versions, and your preferred download location. The terminal must be running as admin if the destination bin folder is in a location requiring privileged access.

# If you downloaded zstd above, make sure your working folder is the same as where the zstd binary is located
working="/c/temp-downloads"
# Set the location of the Git usr/bin folder (under normal installations we can simply use /usr/bin because the MSYS2 root is the Git installation folder)
git_usr_bin_folder_location="/usr/bin"
rsync_url="https://repo.msys2.org/msys/x86_64/rsync-3.2.7-2-x86_64.pkg.tar.zst"
libxxhash_url="https://repo.msys2.org/msys/x86_64/libxxhash-0.8.1-1-x86_64.pkg.tar.zst"
mkdir -p "$working"
curl -L -o "$working/rsync.tar.zst" "$rsync_url"
curl -L -o "$working/libxxhash.tar.zst" "$libxxhash_url"
tar -I "$working/zstd.exe" -xvf "$working/rsync.tar.zst" --directory="$working"
tar -I "$working/zstd.exe" -xvf "$working/libxxhash.tar.zst" --directory="$working"
cp -a "$working/usr/bin/." "$git_usr_bin_folder_location"

That should be all you need to get rsync working as everything else necessary should already be included in the base Git for Windows MSYS2 environment. You can now clean up the contents of the working folder. Hopefully at some point zstd will be included in the base environment and the extra step of acquiring it won't be needed, but it is what it is for now.

If you want to also use Pageant authentication with rsync over ssh, the ssh-pageant binary is already installed in Git for Windows and all you need to do is add a small bit to your ~/.bashrc or ~/.bash_profile file (see ssh-pageant usage section and Git for Windows wiki). I've modified it specifically for use with the Git for Windows environment and enhanced it with some detection for a missing socket file:

# ssh-pageant allows use of the PuTTY authentication agent (Pageant)
SSH_PAGEANT="$(command -v ssh-pageant)"
if [ -S "$SSH_AUTH_SOCK" ]; then
  PAGEANT_SOCK=$SSH_AUTH_SOCK
else
  PAGEANT_SOCK=${TEMP:-/tmp}/.ssh-pageant-$USERNAME
fi
if [ -x "$SSH_PAGEANT" ]; then
  eval $("$SSH_PAGEANT" -qra "$PAGEANT_SOCK")
fi
unset SSH_PAGEANT
unset PAGEANT_SOCK
18
  • 1
    thank you sir, downloading and it works .. who would have known...
    – pscheit
    Commented Nov 18, 2017 at 7:46
  • 8
    An update: since 2020, there's one more step required first. The rsync packages in that repo are no longer in .tar.xz format but rather .tar.zst, and Git for Windows out of the box doesn't understand that format. Happily, it's enough to first install zstd: download zstd-….pkg.tar.xz from that same repository, unpack and install that, and then tar -xvf is able to handle the rsync.pkg.tar.zst file.
    – Greg Price
    Commented Mar 16, 2021 at 21:19
  • 5
    Further to Greg Price's info relating to zstd now being required: you cannot pick the latest zstd package because it is bundled in zstd format (chicken and egg, much?) but there is an older version with .xv. Also my experience was that calling rsync having put zstd and rsync in /usr/bin results in rsync --version complaining that msys-zstd-1.dll is not found, so unpacking-the-zstd-package-into-\usr\bin instruction is incomplete Commented Aug 2, 2021 at 22:08
  • 4
    I found it was necessary to d/l rsync, libxxhas, liblzr, and libzstd, unpack with 7-Zip-zstd, and copy into /usr/bin Commented Aug 2, 2021 at 22:40
  • 2
    I was able to get rsync working with only libxxhas and libsztd. But I did the untaring in wsl
    – CervEd
    Commented Oct 10, 2021 at 13:36
7

I know the question isn't "Using rsync for MSYS", but I think mentioning this is relevant:

"MinGW Shell" (formerly best known as "MSYS (Basic System)") does indeed have rsync. I'm using that shell/system as often as possible since it's much smaller than Cygwin. (I'm probably the only one that cares about file sizes these days.)

"Git for Windows" / "Git Bash" could easily add rsync since it based on MinGW/MSYS and it has been mentioned on the mailing list earlier.

PS! If your MinGW Shell doesn't have rsync, just run "mingw-get install msys-rsync" to add it.

Updated 25th of March 2016:

Git for Windows now uses MSYS2. And MSYS2 uses "pacman", not "mingw-get", to install packages. Unfortunately, "Git for Windows" doesn't include pacman ... There are two options:

  1. Install the Git for Windows SDK (which includes pacman).
  2. Install Git inside MSYS2.

The last option is my favorite, but it's not streamlined yet so the first option is probably easier. Hopefully, it will be fixed.

PS! I haven't played much with MSYS2 / pacman, but it seems very promising.

Updated 7th of August 2019:

If you are using Git for Windows (Git Bash) it's much better to just grab the rsync package from the MSYS2 Package repository, as is described in the answer above.

6
  • sadly, in Git Bash: $ mingw-get bash: mingw-get: command not found Commented Mar 24, 2016 at 17:06
  • Yes, it's a bit sad. Look at my updated answer above - still sad, but the situation has improved.
    – hansfn
    Commented Mar 25, 2016 at 8:40
  • I went for the second option and I ended up with a 2.17GB folder, but everything work like a charm. I installed rsync by using pacman -S rsync. In the past I used CygWin but I think MSYS2 is awesome, I don't have to run a gui to install extra packages.
    – Edenshaw
    Commented Jun 21, 2016 at 16:39
  • The bug report listed in this answer has since been closed as "unfixable" - however, I have created a pull request to have rsync included with Git for Windows. Commented Feb 14, 2017 at 12:54
  • Any links on how to do either option 1 or option 2? I think I have Git for Windows SDK. Dunno. But assuming I do, or can get it.. what then? If not, I have no idea what it even means to install Git inside MSYS2. Does that mean I need to delete my current Git install. Will I loose anything, like settings or functionality if I do so. Will nay links break? Commented Jul 23, 2019 at 14:22
2

I assume by git bash you really mean msysgit, aka "git for windows." The Start Menu option is git bash, which explains the name you used..

First, you are correct in dismissing git as an option here. One of the things git does not do well is handling binary files--and you'd end up with a huge, bloated repository. Additionally, since you won't be handling changes, using the overhead of git strikes me as overkill.

Secondly, it sounds like the question you're actually asking is, "Can I use rsync from msysgit/Windows?" The short answer is, not out of the box. msysgit is really just the Windows command line in terms of form and function. That being said, you have two options: download a third-party tool to enable rsync on your Windows machine (Stack Overflow: Rsync for Windows), or just use cygwin. Really, they have the same outcome--so its up to you to decide which ones work better.

3
  • I can use rsync from my cygwin shell, but not from msysgit. I tried copying rsync.exe from cygwin's bin folder over to msysgit's bin folder, but it still isn't finding it. Is there some other step I need to do to install it there? It's strange that I'm getting an error that it can't find the file, instead of giving me an error that one of the file's dependencies is missing. Thanks, Will
    – Will Gant
    Commented Sep 13, 2011 at 1:31
  • This simply isn't possible. cygwin sets up a GNU-based environment, including Linux-like libraries, configs, etc.. msysgit is the Windows command line, which supports Windows executables. What's likely happening is, its not even able to detect the file as executable. Simply put, cygwin and msysgit are not even remotely compatible.
    – Andrew M.
    Commented Sep 13, 2011 at 2:08
  • It is possibly but hacky. Copy cygiconv-2.dll, cygwin1.dll and rsync.exe from Cygwin to your msysgit path. Nothing else (especially not ssh.exe). The problem is usually an incompatibility between Git SSH and Cygwin/rsync SSH. Try to keep the same version. You'll still have issues with the /cygdrive/ but there I've no solution except a cd.
    – Wernight
    Commented Jun 18, 2012 at 23:47
2

Update January 2021:

Since mysys2 is now using zstd you need to download more than just rsync to get this working. As well as rsync-3.2.3-1-x86_64.pkg you also need to download and copy libzstd-1.4.8-1-x86_64.pkg libxxhash-0.8.0-1-x86_64.pkg and for completeness I also downloaded zstd-1.4.8-1-x86_64.pkg

and 7-zip doesn't do the job of extracting the .zst compressed files wither, so I had to get PeaZip.

3
  • ..except that it seems to be unusably slow! Commented Jan 3, 2021 at 17:40
  • Aha. I have a network drive mounted as P: so I can rsync to it either as /p/mycopyfolder/ or 192.168.0.14::/mycopyfolder/ The /P/ method is unusably slow whereas explicitly using the IP address is as fast as I would expect. Interestingly I get exactly the same effect when using cygwin64 instead of mysys2 : the IP address version is fast but /cygdrive/p/ is unusably slow. msys2: rsync version 3.2.4dev protocol version 31 cygwin64: rsync version 3.2.3 protocol version 31 Commented Jan 3, 2021 at 23:37
  • Thank you for this 2021 update! To unpack the .zst files you could also install 7-Zip Zstandard Edition And this link explains how you can run rsync directly from a command prompt window. Commented Mar 21, 2021 at 3:41
0

For what it's worth, I was having similar problems getting cwRsync to work in msysgit when dropped into the bin path, but after trying lots of different things eventually found that simply updating to the latest version (2.10.2) of Git for Windows fixed the issue.

2
0

rsync is a great for unidirectional sync. You can either run rsync under cygwin, or for Windows there is also Deltacopy

If you wanted to have bidirectional sync, then take a look at unison

For a cross platform directory synchronisation tool, there is syncthing which runs on most platforms.

5
  • unison is great, but you have to install it on server - like 90% of servers are rsync ready. And OP asked for rsync
    – pscheit
    Commented Nov 18, 2017 at 7:50
  • the OP asked about rsync, but the what the OP was trying to achieve was "to be efficiently synchronized between boxes", and unison is available in most repos... Commented Nov 19, 2017 at 11:20
  • Looks like DeltaCopy is not available for Windows 10. May no longer be maintained? Commented Jul 23, 2019 at 17:48
  • It's difficult to tell if Unison works on Windows 10. Hasn't had an update in a long while. There is no mention of what versions of Windows OS that this software supports. There are several issues re: Windows 10 github.com/bcpierce00/unison/… Commented Jul 23, 2019 at 17:54
  • As a cross platform tool to keep a directory structure in sync these days, I'd suggest syncthing syncthing.net However in 2011.... Commented Jul 24, 2019 at 20:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .