1

I am using rsync to copy one big file (> 4 GB) from hard disk to usb drive with "-av --progress" option. Before that, the file already exists on the usb and is the same as the one on disk. So basically I am doing a test. My assumption is, if rsync is smart enough it shouldn't do the real copy.

However, it still tries to copy (at least from the progress and usb blinking). This is the first thing that I don't understand.

Then it complains not enough space (the free space on usb is 2 GB). This shouldn't happen because it only needs to replace the file which takes same amount of space, right? I even use "--delete-before" and it still complains.

Thank you for the hint.

Improve this question
2
  • Try adding --itemize-changes to your command. BTW, what filesystem is the drive? You aren't trying to put a file larger than 4GB on a fat32 filesystem are you?
    – Zoredache
    Sep 17, 2014 at 20:49
  • 1
    Maybe the USB blinking means that rsync is comparing the file contents... you can perhaps try to strace rsync to see what system calls it does and if it is actually reading from, or writing to the USB drive (I'm assuming that you run rsync on Linux, is that the case?)
    – Ale
    Sep 17, 2014 at 20:54

1 Answer 1

Reset to default
1

To your first issue, what is the filesystem on the USB stick? It may not contain enough information about the file for rsync to make determinations about timestamps, permissions, etc.

As to your second point, the --partial option causes rsync to create a temporary file while copying which then gets renamed once the transfer is finished.

For instance, I'm running this command right now on on a Ubuntu 14.04 system:

/usr/bin/rsync -avzh --partial --progress 192.168.42.40:/Users/Shared/bsd /srv/share/distros

This is producing the following output:

receiving file list ... 
36 files to consider
bsd/
bsd/FreeBSD-10.0-RELEASE-i386-dvd1.iso
        738.41M  31%   11.20MB/s    0:02:17

But the file being created is:

15:28:57 file01 lnelson:/srv/share/distros/bsd> ls -al
drwxr-xr-x 3 lnelson lnelson       4096 Sep 17 15:28 .
drwxr-xr-x 4 lnelson lnelson       4096 Sep 14 23:12 ..
-rw------- 1 lnelson lnelson  242745344 Sep 17 15:28 .FreeBSD-10.0-RELEASE-i386-dvd1.iso.Xo0LH8

When the operation completed, I got:

15:28:58 file01 lnelson:/srv/share/distros/bsd> ls -al
drwxr-xr-x 3 lnelson lnelson       4096 Sep 17 15:31 .
drwxr-xr-x 4 lnelson lnelson       4096 Sep 14 23:12 ..
-rw-r----- 1 lnelson lnelson 2317000704 Sep 15 23:36 FreeBSD-10.0-RELEASE-i386-dvd1.iso

If rsync is indeed recopying the file and you are using the --partial flag, then it is creating a temporary new file that has a . as the first character so you'll only see it if you do an ls -a or ls -al while the rsync is in progress to see the temporary file. This means you need enough free space on the USB drive to handle the exiting 4GB copy plus the new 4GB temporary file until the copy finishes, at which time the temporary will replace the original and you'll be back to only using 4GB on the USB drive. During the rsync, you'll need 8GB of storage for the operation to complete.

Improve this answer
1
  • Use --inplace to update the file instead of using a temp file which then replaces the original. I still suspect fat32 of being the problem though.
    – wurtel
    Sep 18, 2014 at 7:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .