I have a large directory of files and I would like to maintain a single compressed tar archive that I can server over http containing all the files which is brought up to date once each day.

Many of the files will not change day-to-day and I'd like to avoid hours of processor time compressing the same files every day.

tar "cannot update compressed archives", so tar uj won't help.

Is there a clever way to do this?


Because this is accessed over http, use PHP to generate the tar file on the fly using something like this:

    header("Pragma: public"); // required
    header("Expires: 0");
    header("Cache-Control: must-revalidate, post-check=0, pre-check=0");
    header("Cache-Control: private",false); // required for certain browsers
    header("Content-Type: application/x-bzip2; charset=binary");
    header("Content-Disposition: attachment; filename=\"archive.tar.bz2\";" );
    passthru("tar cj --exclude-vcs /path/to/files",$err);
    if ($err) {
        error_log("exit value: $err");

Obviously this does not help with the case that this file is going to be downloaded a lot.

| improve this answer | |

This idea was on a competitor advice forum.

tar cf --newer YYYYMMDD

    To copy differences or only the files since the last tar date 
    here is the command.

    Let's say we did a tar on Feb 9th, 2009 as follows:
    (cd /mydata; tar cf - *) | tar xvf -

    Today is Feb 11th and we only want to copy the files that have changed 
    since Feb 9th, 2009. The command would be
    (cd /mydata; tar cf - --newer 20090209 * ) | tar xvf - 
| improve this answer | |
  • Interesting read, thanks, but that is not maintaining an up-to-date compressed archive. – artfulrobot Nov 18 '13 at 11:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.