Different md5sums for same tar contents

Question

I run a test creating two tars from the same dir (its files remained unchanged), and I found that their md5sums were different. I assume there's some timestamp being included in the tar's header, but I haven't found the way to override it. My OS is Ubuntu 9.1. Any ideas ?

Thanks.

Answer 1

As Dennis pointed out above, it's gzip. Part of the gzip header is a mod time for whatever is compressed in the file. If you need gzip, you can compress the tarfile as an extra step outside of tar rather than using tar's internal gzip. The gzip command has a flag to suppress the saving of that modification time.

tar -c ./bin |gzip -n >one.tgz
tar -c ./bin |gzip -n >two.tgz
md5sum one.tgz two.tgz

This will not affect times inside the tarfile, only the one in the gzip header.

Answer 2

I had this problem too, to make gzip do not alter the timestamp, use gzip -n

-n, --no-name do not save or restore the original name and time stamp

[valter.silva@alog ~]$ gzip --help
Usage: gzip [OPTION]... [FILE]...
Compress or uncompress FILEs (by default, compress FILES in-place).

Mandatory arguments to long options are mandatory for short options too.

  -c, --stdout      write on standard output, keep original files unchanged
  -d, --decompress  decompress
  -f, --force       force overwrite of output file and compress links
  -h, --help        give this help
  -l, --list        list compressed file contents
  -L, --license     display software license
  -n, --no-name     do not save or restore the original name and time stamp
  -N, --name        save or restore the original name and time stamp
  -q, --quiet       suppress all warnings
  -r, --recursive   operate recursively on directories
  -S, --suffix=SUF  use suffix SUF on compressed files
  -t, --test        test compressed file integrity
  -v, --verbose     verbose mode
  -V, --version     display version number
  -1, --fast        compress faster
  -9, --best        compress better
    --rsyncable   Make rsync-friendly archive

With no FILE, or when FILE is -, read standard input.

Report bugs to <bug-gzip@gnu.org>.

Example:

[valter.silva@alog ~]$ ls
renewClaroMMSCanaisSemanal.log.gz  s3

[valter.silva@alog ~]$ gunzip renew.log.gz 
[valter.silva@alog ~]$ gunzip s3/renew.log.gz 

[valter.silva@alog ~]$ md5sum renew.log 
d41d8cd98f00b204e9800998ecf8427e  renew.log

[valter.silva@alog ~]$ md5sum s3/renew.log 
d41d8cd98f00b204e9800998ecf8427e  s3/renew.log

[valter.silva@alog ~]$ gzip -n renew.log 
[valter.silva@alog ~]$ gzip -n s3/renew.log 

[valter.silva@alog ~]$ md5sum renew.log.gz 
7029066c27ac6f5ef18d660d5741979a  renew.log.gz

[valter.silva@alog ~]$ md5sum s3/renew.log.gz 
7029066c27ac6f5ef18d660d5741979a  s3/renew.log.gz

Answer 3

To make a tar file with a consistent checksum, just prepend GZIP=-n like this:

GZIP=-n tar -zcf myOutputTarball.tar /home/luke/directoryIWantToZip

How this works: Tar can accept gzip options using a temporary GZIP environment variable, as above. Like Valter said, tar uses gzip, which by default puts a timestamp in the archive. This means you get a different checksum when you compress the same files. The -n option disables that timestamp.