This is a crosspost from stackoverflow because I find that I might get a much qualified audience here
I have a pretty large Subversion repository that I am trying to backup efficiently. The repository size is about 6 GB and growing. Some large commits are around 500 to 1GB in size.
I am trying to backup this repository to an off-site location, through an Internet up-link.
Explaining the sheer size of it
To whomever is wondering, the whole production environment for various sites (config files, EXEs, data files) is kept in this one repository so that we can rollback to an existing working version and track the changes to the production setup. Code is kept on a different repository.
Here is what I am actually doing:
- Backup of the repository to a working folder on the server using the "svnadmin hotcopy SRCDIR TGTDIR"
- Encrypt and compress that repository using "rsyncrypto -r SRCPATH DSTPATH KEYSPATH CERTIFICATE"
- Backup that encrypted version to an off-site location using "rsync -Crtv" (actually cwRsync because I am running on Windows)
First I have to say that it works, though it still has an underlying issue.
The problem lies with the fact that I was expecting that each time the process would run, only the new revision files/data would be copied ([repos]/db/revs/0/...) thus requiring only bandwidth and time when a large commit is made. However, instead:
- If I run only step #3 many times, rsync behaves as it should and nothing is copied because nothing has changed.
- If I run only steps #2 & #3 many times, rsync also behaves well. The envrypted version is the same everytime and rsync doesn't have to transmit anything.
- But, it seems that every time I run all three steps (with a new commit having been made to the repository) the whole repository is being re-uploaded in full. Thus, defeating the whole purpose of using rsync in the first place.
It is as though the files in [repos]/db/revs/0/... are changing everytime I make a hotcopy.
Is this an expected behavior from "svnadmin hotcopy" that the [repos]/db/revs/0/... are changing from one hotcopy to another?
Any suggestion or options I could use to make this hotcopy rsync friendly or say rsyncable?
I am not quite sure that the use of 'svnadmin dump' on the whole repository would produce an "rsyncable" file.