Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Assume an SVN setup that consists of a single repo with nested project folders, something like this:

\
 - trunk
 - - projectA
 - - - trunk
 - - - - ...
 - - - branches
 - - - - ...
 - - projectB
 - - - trunk
 - - - - ...
 - - - branches
 - - - - ...
 - ...

The whole repo is about 40GB on disk, with 17605 commits at last count. I now need to extract a single project from the repo to setup on a separate SVN server, which I understand is possible only by using svnadmin dump on the entire repository and then using svndumpfilter to isolate the project that I need. I fully expect that this will take a really long time to complete the initial dump. Is there a good formula for calculating exactly how long it will take and how much disk space will be required for the final dump file? Also, I've heard that the dump operation uses 100% CPU while it runs. Is that true?

Alternately, is there a better way to go about this given the size of the repo? (Other than just doing an export and losing the revision history.)

share|improve this question
1  
Why wouldn't it use 100% cpu when it is running. If the processor isn't doing anything else, and you are running a program, then you want to use all your resources to make it complete faster. You could set the priority of your dump if other important services are running on the system. On some *nix there is a command nice to start a process with a lower priority. –  Zoredache Sep 8 '11 at 17:54
add comment

2 Answers 2

up vote 0 down vote accepted

Is there a good formula for calculating exactly how long it will take and how much disk space will be required for the final dump file?

To avoid the disk I/O, you can pipe the svndumpfilter result to netcat.

On the old SVN server:

svnadmin dump /path/to/your/repo/ | svndumpfilter --drop-empty-revs --renumber-revs include single_project | nc -l 2302

and on the new SVN server:

svnadmin create single_project
nc IP_address 2302 | svnadmin load single_project

I've tested with my repo (4GB, ~12000 revisions), it takes ~12 minutes to complete.

PS: You can also use gzip to compress the data and ionice to run this with low priority.

share|improve this answer
    
I should have mentioned that the source machine is Windows. I don't think I will have command line access to the new machine as it is a hosted SaaS repo. –  Chrisbloom7 Dec 8 '11 at 18:05
    
Thanks for providing this answer. I'm marking this as the answer because theoretically I think it would have done exactly what I needed even though the project scope changed and I didn't need to do this after all. –  Chrisbloom7 Jan 10 '12 at 15:24
add comment

That'll be pretty massive without the --deltas option, which you can't use when you're going to filter it. The time and space needed will, of course, depend on your server's performance and the contents of the repo.

If you need to do a dry run to see the space usage, just send it to /dev/null - and as @Zoredache mentioned, set the process's niceness as needed.

A couple recommendations:

  • Send it straight to a compression engine from stdout; they compress well.
  • If the size usage is too much to handle (which I suspect it might be), do it in parts.
    • dump the first thousand revisions (svnadmin dump -r 1:1000),
    • filter them,
    • load into the new repo,
    • delete the first dump, make a new one, rinse and repeat.
share|improve this answer
    
Thanks for this comment. It sounds logical, but the client has since asked for a simple export of trunk and they will be starting a new Git repo with it instead, so I won't be able to test your suggestion. –  Chrisbloom7 Dec 8 '11 at 18:04
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.