Google Cloud Storage - Different file via http vs https

Question

I am facing some really weird bug with Google Cloud Storage.

I uploaded a 1MB zero-filled file to a bucket, and whenever i download it with http i get a different file (Different contents, different md5, same size) than if i download it with https (which gets me the correct file).

The real MD5 is: b6d81b360a5672d80c27430f39153e2c

Example: (You can try it at home)

wget http://storage.googleapis.com/weirdbug/test1m.test
md5 test1m.test
// MD5 (test1m.test) = 0575922c3eda2ae6fb2865cc136f65ee
rm test1m.test
wget https://storage.googleapis.com/weirdbug/test1m.test
md5 test1m.test
// MD5 (test1m.test) = b6d81b360a5672d80c27430f39153e2c

I want to believe that i am doing something wrong on my end, but i can't seem to figure it out.

Any ideas? I thought that GCE Storage should work the same with both HTTP & HTTPS.

Answer 1

The correct MD5 sum for a one mebibyte file composed entirely of zero bytes is b6d81b360a5672d80c27430f39153e2c (the one you saw for HTTPS), so something is up with your HTTP wget. I'm wondering if perhaps you have a proxy or something that is inserting junk into your response, which doesn't happen for your HTTPS request.

Here's what I see when I try this:

$> dd if=/dev/zero of=/tmp/test1m.test bs=1024 count=1024
1024+0 records in
1024+0 records out
1048576 bytes (1.0 MB) copied, 0.00866687 s, 121 MB/s
$> md5sum /tmp/test1m.test 
b6d81b360a5672d80c27430f39153e2c  /tmp/test1m.test
$> gsutil cp -a public-read /tmp/test1m.test gs://special-test/
Copying file:///tmp/test1m.test [Content-Type=application/octet-stream]...
Uploading   gs://special-test/test1m.test:                     1 MiB/1 MiB      
$> wget http://storage.googleapis.com/yarbrough-test/test1m.test
$> md5sum test1m.test
b6d81b360a5672d80c27430f39153e2c  test1m.test