I'm using rsync to upload/download a pretty large file (20Gb). I'm aware of --progress option, but what it does in my case is just showing me the result at the end.
My problem is that during some unstable connection - I don't know whether it's still downloading/uploading or something went wrong. Is there a way to show the progress bar for an individual file? Something similar to what wget does by default?
Thanks!
This might be what you're after.
% rsync -avvz --times --stats --checksum --human-readable --acls \
--itemize-changes --progress \
--out-format='[%t] [%i] (Last Modified: %M) (bytes: %-10l) %-100n' \
/usr/include/glib-2.0 my-glib-copy/
The switches breakdown as follows:
NOTE: See the man pages for rsync and rsyncd.conf for full details on the above switches.
The above command produces the following transcript:
sending incremental file list
delta-transmission disabled for local transfer or --whole-file
[2012/12/23 21:34:46] [cd+++++++++] (Last Modified: 2010/12/19-08:13:31) (bytes: 4096 ) glib-2.0/
[2012/12/23 21:34:46] [>f+++++++++] (Last Modified: 2010/09/30-15:02:30) (bytes: 1511 ) glib-2.0/glib-object.h
1.51K 100% 0.00kB/s 0:00:00
1.51K 100% 0.00kB/s 0:00:00 (xfer#1, to-check=181/183)
[2012/12/23 21:34:46] [>f+++++++++] (Last Modified: 2010/09/30-15:02:30) (bytes: 2910 ) glib-2.0/glib.h
2.91K 100% 2.78MB/s 0:00:00
2.91K 100% 2.78MB/s 0:00:00 (xfer#2, to-check=180/183)
[2012/12/23 21:34:46] [>f+++++++++] (Last Modified: 2010/09/30-15:02:31) (bytes: 3613 ) glib-2.0/gmodule.h
3.61K 100% 3.45MB/s 0:00:00
3.61K 100% 3.45MB/s 0:00:00 (xfer#3, to-check=179/183)
...
...
[2012/12/23 21:34:46] [>f+++++++++] (Last Modified: 2010/09/30-15:02:31) (bytes: 8431 ) glib-2.0/gobject/gvaluecollector.h
8.43K 100% 141.96kB/s 0:00:00
8.43K 100% 141.96kB/s 0:00:00 (xfer#178, to-check=1/183)
[2012/12/23 21:34:46] [>f+++++++++] (Last Modified: 2010/09/30-15:02:31) (bytes: 8507 ) glib-2.0/gobject/gvaluetypes.h
8.51K 100% 143.23kB/s 0:00:00
8.51K 100% 143.23kB/s 0:00:00 (xfer#179, to-check=0/183)
total: matches=0 hash_hits=0 false_alarms=0 data=1305506
rsync[2996] (sender) heap statistics:
arena: 540672 (bytes from sbrk)
ordblks: 3 (chunks not in use)
smblks: 7
hblks: 2 (chunks from mmap)
hblkhd: 401408 (bytes from mmap)
allmem: 942080 (bytes from sbrk + mmap)
usmblks: 0
fsmblks: 592
uordblks: 404784 (bytes used)
fordblks: 135888 (bytes free)
keepcost: 134240 (bytes in releasable chunk)
rsync[2999] (server receiver) heap statistics:
arena: 286720 (bytes from sbrk)
ordblks: 2 (chunks not in use)
smblks: 8
hblks: 3 (chunks from mmap)
hblkhd: 667648 (bytes from mmap)
allmem: 954368 (bytes from sbrk + mmap)
usmblks: 0
fsmblks: 672
uordblks: 174480 (bytes used)
fordblks: 112240 (bytes free)
keepcost: 102352 (bytes in releasable chunk)
rsync[2998] (server generator) heap statistics:
arena: 233472 (bytes from sbrk)
ordblks: 4 (chunks not in use)
smblks: 6
hblks: 2 (chunks from mmap)
hblkhd: 401408 (bytes from mmap)
allmem: 634880 (bytes from sbrk + mmap)
usmblks: 0
fsmblks: 448
uordblks: 83152 (bytes used)
fordblks: 150320 (bytes free)
keepcost: 131120 (bytes in releasable chunk)
Number of files: 183
Number of files transferred: 179
Total file size: 1.31M bytes
Total transferred file size: 1.31M bytes
Literal data: 1.31M bytes
Matched data: 0 bytes
File list size: 6.30K
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 350.39K
Total bytes received: 3.43K
sent 350.39K bytes received 3.43K bytes 707.64K bytes/sec
total size is 1.31M speedup is 3.69
I came across this SF question:
One of the answers mentioned that in newer versions of rsync there is now support for a progress bar. It looks like this:
% ./rsync -a --info=progress2 /usr .
305,002,533 80% 65.69MB/s 0:00:01 xfr#1653, ir-chk=1593/3594)
This progress bar still doesn't show the status of individual files in the transfer, but it does show a percentage of the overall transfer which might be helpful.
rsync -vP works for me.
From the man page: "-P same as --partial --progress"
Are you using rsync in daemon mode? The default behaviour of rsync (on Debian 6) is how you've requested. However, I'm finding in the man page for rsyncd.conf that the default behaviour in daemon mode is:
transfer logging
This parameter enables per-file logging of downloads and uploads in a format somewhat similar to that used by ftp daemons. The daemon always logs the transfer at the end, so if a transfer is aborted, no mention will be made in the log file.
If you want to customize the log lines, see the "log format" parameter.
So if using rsync in daemon mode, it looks like you'll have to add the line "transfer logging" to rsyncd.conf.
I think Pipe Viewer is what you need: https://code.google.com/p/pipeviewer/
