I have a large directory of files and I would like to maintain a single compressed tar archive that I can server over http containing all the files which is brought up to date once each day.

Many of the files will not change day-to-day and I'd like to avoid hours of processor time compressing the same files every day.

tar "cannot update compressed archives", so tar uj won't help.

Is there a clever way to do this?


2 Answers 2


Because this is accessed over http, use PHP to generate the tar file on the fly using something like this:

    header("Pragma: public"); // required
    header("Expires: 0");
    header("Cache-Control: must-revalidate, post-check=0, pre-check=0");
    header("Cache-Control: private",false); // required for certain browsers
    header("Content-Type: application/x-bzip2; charset=binary");
    header("Content-Disposition: attachment; filename=\"archive.tar.bz2\";" );
    passthru("tar cj --exclude-vcs /path/to/files",$err);
    if ($err) {
        error_log("exit value: $err");

Obviously this does not help with the case that this file is going to be downloaded a lot.


This idea was on a competitor advice forum.

tar cf --newer YYYYMMDD

    To copy differences or only the files since the last tar date 
    here is the command.

    Let's say we did a tar on Feb 9th, 2009 as follows:
    (cd /mydata; tar cf - *) | tar xvf -

    Today is Feb 11th and we only want to copy the files that have changed 
    since Feb 9th, 2009. The command would be
    (cd /mydata; tar cf - --newer 20090209 * ) | tar xvf - 
  • Interesting read, thanks, but that is not maintaining an up-to-date compressed archive. Nov 18, 2013 at 11:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .