Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I currently rsync on a Linux host to copy things from Windows to my Linux box. But I can't copy files with long names.

I have tried DeltaCopy, cwrsync, and cygwin. From what I have found there all of these tools refuse to copy files when the length of the file becomes long, this seems to be somewhere around 255 characters.

This issue is addressed on the cwrsync forums here and it supposedly it may be fixed sometime in the future whenever cygwin 1.7 comes out and UTF8 is supported. There is even a test build here.

I am not particularly comfortable using the test build on a production system. I am hoping someone knows of another rsync option.

As an alternative to rsync do you know of another tool that I can copy a directory structure on Linux from a windows host that will not have problems with long or unusually named files? The important bit is that I need a tool that can easily work across an SSH tunnel. Some of the systems are beyond firewalls, and I believe SSH is the tunnel I will be allowed to use.

share

locked by HopelessN00b Dec 5 at 9:51

This question exists because it has historical significance, but it is not considered a good, on-topic question for this site, so please do not use it as evidence that you can ask similar questions here. This question and its answers are frozen and cannot be changed. More info: help center.

    
FYI cwrsync has released an update that is based on cygwin 1.7 that supports long file names! itefix.no/i2/node/12070 –  Zoredache Oct 12 '09 at 7:14

9 Answers 9

up vote 6 down vote accepted
+175

I may be missing the point but have you considered using Robocopy on Windows. It is similar to RSync however you can't schedule it from the application directly.

This can be overcome by writing a batch file for the copy and then creating a Scheduled Task. Robocopy is free and extremely robust. I often use it to copy files between Linux and Windows using Samba and the network, and the resume capability of Robocopy is really powerful.

share
    
robocopy is an option I have been considering, but it operates in the wrong direction (push from win to lin), I wanted to pull from windows on Linux. I suspect that I may have to go this direction if I can't find anything better though. –  Zoredache Aug 19 '09 at 16:16

Aha! You can use subst with rsync.

If you have a deep directory tree d:\very\long\file\n\ame\etc\etc then experiment reveals that you can subst X: to d:\very\long\file\name\etc and then rsync to and from /cygdrive/x/whatever. This works both at the client end and at the server end.

Now while you can use strategic substs to reduce the length of the filenames I don't know if this allows you to bypass the 260 character limit. It may also not be very convenient. Worth a try though.

John Rennie.

----8<----

See http://www.ratsauce.co.uk/notablog/LongFilenames.asp for my ramblings on long filenames.

I don't know of any way to get the \\?\ prefix to Cygwin, and obviously the current version of Cygwin doesn't use the prefix internally. Presumably they are addressing this in the new version and that's why it will support names longer than 260 characters. I use the Cygwin rsync all over the place so like you I eagerly await the release.

There are some other issues with the Cygwin rsync. It makes a mess of ACLs unless you specify cygwin=nontsec, and it tends to hang on very big directories. On my list of things to do before I die is to write a native Windows version of rsync that won't have these problems. I believe this has been done, but only as commercial versions not public domain ones.

JR

share
    
Rsync appears to hang on large directories because it has to read all the files first. Problem is that windows reads all the files first, then rsync kicks in. It could take windows a few minutes to scan the directory depending on how many files are there. –  Ryaner Aug 20 '09 at 9:47
    
This is no longer the case with rsync 3.0. It now sends an incremental file list so it can start sending files sooner. –  James Sneeringer Aug 24 '09 at 3:40

I would suggest you try Unison http://www.cis.upenn.edu/~bcpierce/unison/

It is a very interesting alternative to rsync because it can also provides bi-directionnal synchronization (which rsync is not able to do). I have already used it successfully across 2 virtual machines and was very happy with the results.

But as far as long filename are concerned I don't know if it works as you want.

From the official website :

Unison is a file-synchronization tool for Unix and Windows. It allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other.

share
    
Have you used it with paths longer the 250 characters? It appears the only way to get it on windows is via cygwin, which I believe means that it will have the exact same issue I am having with the cygwin rsync. –  Zoredache Aug 19 '09 at 16:12
    
I have not tested long path, but Unison binaries for Windows are built using cygwin.dll (it is bundled with the binaries, no need to install cygwin), therefore it might work correctly. Try it :) –  Olivier Jaquemet Aug 20 '09 at 7:18

Just a note that I run into the same problem (limitations of pre-1.7 cygwin) and found a working build of cwrsync with cygwin 1.7 here:

http://www.doering-thomas.de/page.php?seite=1&sub=6&lang=en#rsync

(Original link found from cwrsync forums)

Not exactly an official build, but it solved my charset issues :)

share

http://lists.samba.org/archive/rsync/2009-March/022955.html

http://www.okisoft.co.jp/esc/utf8-cygwin/ Has a UTF8 layer, which has a side effect of increasing the possible path size. According to its author, if you want to use more characters you can up the constant in the patch. That seems even more hackish.

Probably not your best bet for remote sync, but is a windows rsync that claims to support longer file names.

share
    
This post worries me a bit (cygwin.com/ml/cygwin/2006-05/msg00068.html) - "Basically, there's a 930-line patch, most of which is new. FWICS, it may not cover all places where Unicode conversion may be needed (in particular, sending Unicode through a textmode pipe will very likely fail spectacularly)" –  Zoredache Aug 19 '09 at 21:54

So, if the tools are bad on Windows is there a way to use the Linux tool?

With ssh - sshfs?

Without ssh - use a VPN and mount as SMB?

Either way would allow you to use the more capable Linux rsync client against a filesystem. I haven't done this with 10GB of data so YMMV. :)

share
    
I am trying to pull files FROM windows to my Linux box. Plus I really don't want to build a archive since I am talking lots of data 10GB+ that I want to sync weekly. If I build an archive I won't get the benefit of incremental updates. –  Zoredache May 16 '09 at 9:58
    
OK, revised answer. –  Andrew H May 18 '09 at 16:45

This won't do large file structures, but for relatively small data-sets with long names you could look into using something like 7-Zip to create an archive that rsync could then transfer. You said that you need to pull data from Windows servers to a Linux server. If you have shell access, archive the data you need in a container (7-zip) transfer the container, and expand it when it gets to the server. This pushes the long file name problem onto the archiver, for which I believe there is better support and non-Cygwin tools.

share

In order not to lose portability I would recommend you to stay with rsync and try to solve the 255 limitation. This limitation is no longer inside Windows and is in RSYNC code - in fact the limit on Windows is around 2048 now (if I remember well).

I'm pretty sure that there are no other rsync ports than the ones specified by you and I suggest you to use the test version and report any bugs you may find.

If you don't mind using FTP you could try LFTP - it does have a very good mirroring feature but it doesn't compare with rsync when it comes to backup/sync.

share

I hacked together a script that can temporarily rename files or directories with shorter names so that rsync (and other programs) can proceed. You can use this as a part of the tool chain that you are using to copy files from Linux to Windows (rsync or otherwise). I don't know what you mean by "unusual", so perhaps this only addresses a part of your requirements. Please let me know if it is helpful for you.

share

protected by Community Aug 18 at 18:47

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.