1

I'm generating archives which are larger than my current physical media ( DVD ). I'd like to split those archives:

  • automatically - instead of generating mini-archives by hand;
  • consistently - so that an archive can be extracted independently of another.

For instance for a tree of 24GB which would be archived into 10GB I would get 3 archives, all of them < 4.7 GB and each of them being able to be extracted without the other 2.

I'm using dirvish so I'm archiving a filesystem tree.


Update: I'm using Linux.

8 Answers 8

1

I'd be really interested to see how you're combining dirvish with a DVD backup system considering the main point of dirvish is to do backups incrementally, and you're losing that benefit with DVDs.

Anyway, the options for this problem that I can see:

  • Split your tree into smaller trees so that each one fits on a disk. This is probably easiest, but eventually they'll probably get too big again.
  • Script some archiving/compression into post_server, I'd use rar for this, it supports split archives where the maximum segment size is given as a parameter. Getting all the data back if one disk is lost could be difficult. (Using tar as previously suggested may get around this problem)
  • Split the branch without an archiver. It'd be possible to use find to write a shell script that would split out the branch into chunks less than 4.7gb. This is probably the best long term approach, but requires lots more effort

Personally, I'd try to avoid any solutions that involve building large 4.7gb archives If I thought there was a chance I'd have to extract them to get small files back out

0

PKZip and WinZIP both support the option of split a large archive into multiple, smaller sized files that can be user specified.

I'm unfamiliar with dirvish, but if you are looking just to backup and compress the files, with a user defined file size limit, it might be worth considering.

2
  • I'm using Linux - so all I get is zip. Are zipsplit files able to be operated independently? Jun 26, 2009 at 14:54
  • No, they have to be opened together so that would be a problem since you are using DVD media. I did not consider that. Sorry. Jun 26, 2009 at 20:19
0

You can tell tar to split an archive at a given size.

It was originally implemented for splitting archives across multiple-tapes, but equally applicable for other types of media.

You won't be able to initially compress the archive. But you should be able to gzip and later extract the split archives individually. Be sure to specify new names for each split archive though.

tar cvf archive.tgz -ML 4928307 items_to_archive
0

as far as I know you can't create a split archive so that one of its parts can be extracted independently.

You can tell tar, zip, 7zip, rar, ... to split the archive into multiple files specified by size. (rar -v, 7z -v, etc)

I know that RAR can create archives in such a way that you can extract only from one volume from the whole volume set.

0

What you want is unlikely to exist, if only because when doing a restore, the software has to figure out which media the backup is on, so it has to have some kind of directory to look such things up in, and such a directory has to be stored in the archive - which is going to require you to have both the 'directory' disk and the 'storage' disk be read. Even if you were to write it yourself, you'd end up having to at least temporarily (for the length of the backup session) track which files had been backed up so they didn't get backed up twice.

0

Periodically archive (tar/zip/...) your stable (not while running a dirvish job!) dirvish bank/vault then use the "PAR" (currently v2) archive format to split the archive.

It is a sort of "software RAIF" (RAID on files!)

See for example http://parchive.sourceforge.net/

0

GNU tar has multivolume support. Useful for when the archive created is larger than a single tape's capacity. I hope that all of you understand that tape can also mean file, or DVD, or CD, etc.

From section 9.6 of the GNU tar manual:

Use --multi-volume (-M) on the command line, and then tar will, when it reaches the end of the tape, prompt for another tape, and continue the archive. Each tape will have an independent archive, and can be read without needing the other. (As an exception to this, the file that tar was archiving when it ran out of tape will usually be split between the two archives; in this case you need to extract from the first archive, using --multi-volume (-M), and then put in the second tape when prompted, so tar can restore both halves of the file.)

emphasis mine

But do be careful about the exception! Maybe write out to files which you can check for size before writing the archive files to DVD. This could be done by a script. Or make the file size something smaller that 4.7G so that an archive that spans two DVDs never occurs.

-1

On linux/unix you can use split, a command that will split a file in different parts that are all a certain size (10 bytes, 100 MB etc)

1
  • Unfortunately split does not satisfy the "and each of them being able to be extracted without the other 2" criteria.
    – jmtd
    Jun 29, 2009 at 13:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .