-1

When using RSYNC, CP or DD on Linux to a Windows SMB share the file gets to 2.0 GB of 6.3 GB and errors stating file is too large.

This has started since disabling SMB1 on the Windows share. I have re-mapped the share using SMB 2.1 in fstab on Linux.

Linux SERVERNAME 3.8.13-68.2.2.el6uek.x86_64 #2 SMP Tue May 12 15:10:51 PDT 2015 x86_64 x86_64 x86_64 GNU/Linux

rsync  version 3.0.6  protocol version 30
Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.

I have tried --partial-dir and it does place the file in the partial directory but when you attempt to run it again to resume it, the file does not resume. It starts over.

[root@SERVERNAME dpdump]# rsync -av --progress --partial-dir=.rsync-partial /u01/dpdump /BackupFolder
sending incremental file list
dpdump/
dpdump/PD20181119.dmp
  2084536320  31%  113.23MB/s    0:00:39
rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken pipe (32)
rsync: write failed on "/BackupFolder/dpdump/PD20181119.dmp": File too large (27)
rsync error: error in file IO (code 11) at receiver.c(301) [receiver=3.0.6]
rsync: connection unexpectedly closed (303 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.6]

I have tried using block-size and protocol from another SO post, still doesn't work.

[root@SERVERNAME dpdump]# rsync -av --progress --partial-dir=.rsync-partial --block-size=108485760 --protocol=29  /u01/dpdump /BackupFolder
building file list ...
220 files to consider
dpdump/
dpdump/PD20181119.dmp
  1998913536  29%  187.58MB/s    0:00:24
rsync: writefd_unbuffered failed to write 4092 bytes to socket [sender]: Broken pipe (32)
rsync: write failed on "/BackupFolder/dpdump/PD20181119.dmp": File too large (27)
rsync error: error in file IO (code 11) at receiver.c(301) [receiver=3.0.6]
rsync: connection unexpectedly closed (577 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.6]

I have tried append-verify as the manpage recommends but it does not resume the file, it overwrites and starts from 0.

[root@SERVERNAME dpdump]# rsync -av --progress --append-verify  /u01/dpdump /BackupFolder
sending incremental file list
dpdump/
dpdump/PD20181119.dmp
  2147549183  32%   33.88MB/s    0:02:10
rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken pipe (32)
rsync: write failed on "/BackupFolder/dpdump/PD20181119.dmp": File too large (27)
rsync error: error in file IO (code 11) at receiver.c(301) [receiver=3.0.6]
rsync: connection unexpectedly closed (302 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.6]

Trying to do a 1GB file using dd works. Trying to use a 3GB file using dd does not. It stops at 2.1 GB.

# dd if=/dev/zero of=/BackupFoldertest.img count=1 bs=1G
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 13.8795 s, 77.4 MB/s

# dd if=/dev/zero of=/BackupFoldertest.img count=1 bs=3G
0+1 records in
0+1 records out
2147479552 bytes (2.1 GB) copied, 28.625 s, 75.0 MB/s
#

Previous rsync's that worked are shown. Stopped when I disabled SMB1 on the Windows share. The 12/10/2018 ones are the ones where it broke.

enter image description here

11
  • 2
    Put the output of rsync --version in your question. I wonder if it was compiled with 64-bit support. Can you also include the output of uname -a? – Bert Dec 11 '18 at 14:53
  • 1
    Can you just copy the file using cp? also try dd if=/dev/zero of=/BackupFolder/test.img count=1 bs=1G and post the result. – davidbaumann Dec 11 '18 at 15:43
  • @davidbaumann: cp File too large. dd: 1073741824 bytes (1.1 GB) copied, 13.8795 s, 77.4 MB/s. Changed it to 3G from 1G and it stopped at 2.1GB copied. – Travis Dec 11 '18 at 20:33
  • Added your details @Bert – Travis Dec 11 '18 at 20:39
  • 2
    So it's not an rsync problem. – davidbaumann Dec 11 '18 at 22:27
1

Could be a filesystem issue on the target disk (FAT16 format?). On disks, the maximum size of a volume or file and the maximum number of files per volume depend on the file system used to format the volume. https://www.genie9.com/Support/KB/KnowledgeArticle.aspx?KBID=113

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.