Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a free dropbox account (2GB), and I was wondering how the versioning of large files works.

I have a full backup of all my webfiles that sites @ just over 1GB. After the initial upload of 1GB, everytime it syncs will dropbox figure out the delta of the file, or will it have to upload the entire thing again to version it?

It would be cool to always have an up to date version of a large file, but I dont want to kill my bandwidth uploading 1GB everytime.

Is this possible?

Thanks,

share|improve this question

closed as too localized by SvW, RobM, Chris S Apr 11 '13 at 13:01

This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.If this question can be reworded to fit the rules in the help center, please edit the question.

add comment

4 Answers

up vote 29 down vote accepted

Dropbox uses a binary diff algorythm to break down all files into blocks, and only upload blocks that it doesn't already have in the cloud. All of this is done locally on your computer.

Dropbox doesn't just use your files that you have already uploaded, it aggregates everyones files into one database of blocks, and checks each local block hash against that database.

This means that if someone else has uploaded the same file as yourself (say for example, the latest Ubuntu iso), then the upload will seem instant as there is nothing to upload, but if you are updating a file that changes regularly, like your backup file, then only the changes are uploaded. If you upload a totally unique file, then you have to wait for it all to upload.

share|improve this answer
    
Any references to this? It's pretty interesting –  STW Aug 11 '09 at 17:46
1  
The Dropbox team talk about it every now and then in the forums (Arash F especially, although they are very busy these days). –  Moo Aug 11 '09 at 17:48
3  
Does this mean it would only upload changed blocks of an encrypted file (e.g. a TrueCrypt volume) as well? –  Will M Aug 11 '09 at 19:49
1  
Will - yes, I believe quite a few people use TrueCrypt within their Dropbox folders with great success. –  Moo Aug 11 '09 at 19:50
1  
The last part of you answer is no-longer true. After the 'Dropship' debacle, changes were made. It is likely that they still de-dupe internally, but if you put the "windows8.iso" (which, odds are, at least someone has already done) in your folder now, you will have to upload every byte. –  DanO Dec 10 '12 at 20:10
show 2 more comments

For what it's worth, Dropbox claims to create hashes on every 4MB of each file. That way, if you change a contiguous 2MB of a 100MB file, it will likely only need to upload 4MB (or 8MB if you cross into a second 4MB block) to re-sync the file.

The hashes we use are only for the 4MB file chunks

Source: http://forums.dropbox.com/topic.php?id=5502

share|improve this answer
add comment

It's also important to highlight that it doesn't upload your whole file at once when you change it. For example, if you have an unique file weighting 2GB, let's say for an encrypted disk drive you hold (like when you use truecrypt or pgpdisk), and you change just a couple of files inside the encrypted disk, dropbox will only upload the blocks that effectively changed. So, for instance, if you upload your pgpdisk file with 2GB to dropbox, and then you change just let's say 100MB of this 2GB, dropbox will be intelligent enough to detect and update only what have changed. So you don't waste your upload bandwidth uploading stuff that is already there.

Another feature that I saw the dropbox team is working on is to make dropbox to detect another instances of dropbox running on your local network, and sync the information in between them. For example, you have a laptop and a desktop, and both have the same dropbox account, and you update your files on your desktop - and the desktop instantly syncs with the "cloud" - when you plug your laptop in, instead of going to the cloud, dropbox will instead download the diff directly from your desktop computer, and won't waste your download bandwidth. This is still to come - but will be a sweet feature!

share|improve this answer
1  
They released an experimental build with peer to peer sync capability over the weekend. –  Moo Aug 11 '09 at 19:51
1  
Now is a stable feature. –  Wiliam Sep 9 '12 at 23:53
add comment

Here's the truth on the benefits of delta sync. It doesn't matter much.

http://blog.syncplicity.com/blog/2009/07/why-delta-sync-doesn-t-matter.html

share|improve this answer
3  
Lone link is considered a poor answer (see faq) since it is meaningless by itself and target resource is not guaranteed to be alive in the future. It would be preferable to include the essential parts of the answer here, and provide the link for reference. –  j0k Mar 3 '13 at 6:16
add comment

Not the answer you're looking for? Browse other questions tagged or ask your own question.