Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I have a large directory of files and I would like to maintain a single compressed tar archive that I can server over http containing all the files which is brought up to date once each day.

Many of the files will not change day-to-day and I'd like to avoid hours of processor time compressing the same files every day.

tar "cannot update compressed archives", so tar uj won't help.

Is there a clever way to do this?

share|improve this question
    
Related question superuser.com/questions/774777/… –  kubanczyk Jun 29 at 15:32

2 Answers 2

up vote 0 down vote accepted

Because this is accessed over http, use PHP to generate the tar file on the fly using something like this:

<?php
    set_time_limit(1);
    header("Pragma: public"); // required
    header("Expires: 0");
    header("Cache-Control: must-revalidate, post-check=0, pre-check=0");
    header("Cache-Control: private",false); // required for certain browsers
    header("Content-Type: application/x-bzip2; charset=binary");
    header("Content-Disposition: attachment; filename=\"archive.tar.bz2\";" );
    passthru("tar cj --exclude-vcs /path/to/files",$err);
    if ($err) {
        error_log("exit value: $err");
    }
    exit;

Obviously this does not help with the case that this file is going to be downloaded a lot.

share|improve this answer

This idea was on a competitor advice forum.

tar cf --newer YYYYMMDD

    To copy differences or only the files since the last tar date 
    here is the command.

    Let's say we did a tar on Feb 9th, 2009 as follows:
    (cd /mydata; tar cf - *) | tar xvf -

    Today is Feb 11th and we only want to copy the files that have changed 
    since Feb 9th, 2009. The command would be
    (cd /mydata; tar cf - --newer 20090209 * ) | tar xvf - 
share|improve this answer
    
Interesting read, thanks, but that is not maintaining an up-to-date compressed archive. –  artfulrobot Nov 18 '13 at 11:20

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.