Skip to main content
73 votes
Accepted

bzip2 too slow. Multiple cores are avaible

There are many compression algorithms around, and bzip2 is one of the slower ones. Plain gzip tends to be significantly faster, at usually not much worse compression. When speed is the most important, ...
marcelm's user avatar
  • 1,161
46 votes

bzip2 too slow. Multiple cores are avaible

Use pbzip2. The manual says: pbzip2 is a parallel implementation of the bzip2 block-sorting file compressor that uses pthreads and achieves near-linear speedup on SMP machines. The output of ...
ThoriumBR's user avatar
  • 5,332
20 votes

How do I load a sql.gz file to my database? (importing)

If you get an error from zcat, in which the error message contains the file name with an extra suffix .Z, then try using gzcat instead, as described at https://stackoverflow.com/questions/296717/zcat-...
Mark Howells-Mead's user avatar
17 votes

Fastest way to extract tar.gz

If you want to see progress use something like pv. Here is an Example: pigz -dc mysql-binary-backup.tar.gz | pv | tar xf -
Tim Hughes's user avatar
16 votes

Break up a dd image into multiple files

It is my command line: dd if=/dev/sda bs=4M | gzip -c | split -b 2G - /mnt/backup_sda.img.gz It will create 2GB files in this fashion: backup_sda.img.gz.aa backup_sda.img.gz.ab backup_sda.img.gz.ac ...
Aaron's user avatar
  • 161
11 votes
Accepted

How to compress SQL file correctly on CentOS?

The size of your mysqldump file is not going to be the same as your database size as reported by PHPMyAdmin. First, create the dump file with the command: mysqldump -u myuser -p mydb > dumpfile....
Bert's user avatar
  • 3,093
10 votes
Accepted

Enable both brotli and gzip on NGINX?

One of the first articles I found when searching online for nginx and brotli, is this link. It kind of answers your question — I think: Browsers which support brotli send ‘br’ along with ‘gzip’ in ...
Tommiie's user avatar
  • 5,695
9 votes

How to gzip logs created by rotatelogs

You could use rotatelogs option -p to use a program to compress the log after the rotation. (See for reference: https://httpd.apache.org/docs/2.4/programs/rotatelogs.html) -p program If given, ...
Enrique Rivera's user avatar
9 votes

gzip not working on nginx for JSON responses

Ah man - I can't believe how dumb this is!!! The data in question was 245 bytes: Content-Length: 245 ...and I had this set: gzip_min_length 256k set. I've changed it to: gzip_min_length 128; and ...
Andrew Newby's user avatar
  • 1,184
8 votes

bzip2 too slow. Multiple cores are avaible

Google's brotli is a newer format that has gained some wide support within browsers recently, as it has some impressive compression, some impressive speed, and perhaps the most impressive combination/...
TOOGAM's user avatar
  • 359
8 votes
Accepted

Does Nginx compress files for every request? (with gzip_on)

Yes Nginx will compress index.html each time it is requested since there is no index.html.gz file. To be honest gzip is not very processor intensive these days and gzipping on the fly (and then ...
Barry Pollard's user avatar
7 votes

How do I load a sql.gz file to my database? (importing)

On macOS, I used this: zcat < [Database].sql.gz | mysql -u root -p [Database Name in MySQL] Enter your password, and voila!
Qasim's user avatar
  • 171
6 votes
Accepted

nginx gzip Vary header together with existing Vary headers

RFC2616 states duplicate headers are acceptable and should be treated the same as a single header with a comma-separated list of values: Multiple message-header fields with the same field-name MAY ...
Jason Martin's user avatar
  • 5,113
6 votes

Enabling Brotli compression on IIS

It appears the Brotli module you referenced requires a paid license, so I haven't tried it, but I encountered a similar issue with my own open source Brotli plugin for IIS. Current browsers advertise ...
saucecontrol's user avatar
6 votes
Accepted

Compress old emails with dovecot

The configuration to compress emails is a two step process: We need to configure dovecot to be able to read compressed emails We need to setup a cronjob to compress emails Dovecot configuration ...
Tofandel's user avatar
  • 201
5 votes

How to zip files with a size limit?

Here's how I did it for a 5GB file (split into 1GB vs 10MB as OP asked)... Example: To split a 5GB file into 1GB files to copy to a FAT32 USB (filename "FIVE_GB_FILE.ISO") Step 1: zip the file (no ...
Jay Marm's user avatar
  • 161
5 votes

How do I load a sql.gz file to my database? (importing)

You can use -c, --stdout, --to-stdout option of gunzip command for example: gunzip -c file.sql.gz | mysql -u root -p database
Ali Jazayeri's user avatar
4 votes

What is the best way to compress backend to nginx reverse proxy data?

http://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_http_version - the HTTP request version required for gzip to compress responses on the back-end is by default 1.1. http://nginx.org/en/docs/...
Tit Petric's user avatar
4 votes

mysqldump to a tar.gz

Try this: mysqldump --all-databases --password=dbpassword | gzip -c | ssh user@servername "cat >/tmp/filename_of_your_choice.gz" Please not that I am in no way good at these things, I just ...
Charlie Candergart's user avatar
4 votes

How do I load a sql.gz file to my database? (importing)

If you are using small size database it's better to extract and import. Here is the extract command tar -xf dbname.sql.tar.gz Here is importing command. mysql -u username -p new_database < data-...
Abu Bakar Iqbal's user avatar
4 votes

gzip piped output into ssh command

adding this to ~/.ssh/config ssh will gzip on the fly :) http://linux.die.net/man/5/ssh_config for more detailed options Host * Compression yes # from 1 (fast) to 9 (slow, best). The default ...
Antony Gibbs's user avatar
4 votes
Accepted

nginx : enabling gzip compression based on hostname

Absolutely if you take a look at the documentation you can see that it is acceptable to place gzip in any of the following levels http, server, location, if in location and as the domain is defined at ...
Drifter104's user avatar
  • 3,833
4 votes

nginx : enabling gzip compression based on hostname

Yes, it is. You should have a server block for each host name, just use gzip on; or gzip off; Inside each server block. The documentation can be found here. The only snag is if you've used some ...
Tim's user avatar
  • 33.1k
4 votes

Nginx not sending gzipped assets when passing trough a load balancer

There is another way to solve this. You can set a higher HTTP protocol version for proxying connections to upstream. It can be done by this parameter: proxy_http_version 1.1; It would be a better ...
mahyard's user avatar
  • 239
4 votes
Accepted

Nginx not sending gzipped assets when passing trough a load balancer

Solved! I figured out myself that the request sent from the load balancer to the upstream (application server), is done in HTTP/1.0, while the nginx server in the application side, only compressed ...
Iván Guillén's user avatar
4 votes

How to replace gzip with pigz without breaking anything

Better method: Tell your backup solution to use pigz instead. Don't try to replace it this way. This might break other things that rely on gzip actually being gzip (unless the pigz developer ...
Sven's user avatar
  • 99.8k
4 votes

How to replace gzip with pigz without breaking anything

Yes, you can. pigz is nearly a a drop-in replacement for gzip. As always, you must test your restores, especially because you changed something. "Replacement" can be a symlink earlier in $PATH. The ...
John Mahowald's user avatar
4 votes
Accepted

Apache GZIP for images - any reason?

Adding gzip compression (with default settings) to that particular file does make it slightly (± 1.7 percent) smaller when I compare the 1019174 bytes for the original file versus 1001991 bytes for ...
HBruijn's user avatar
  • 81.1k
3 votes
Accepted

Is there such thing as unzip cache?

Have you actually looked into the directory to see what's in there? By default, wget won't overwrite existing files, but instead append a number to the name (index.html.1 etc), so your script is ...
Sven's user avatar
  • 99.8k
3 votes
Accepted

Enable compression for HTTP2 server-pushes on an apache server

It seems like this is simply a bug in mod_http2, but it was fixed with version 1.2.6: https://github.com/icing/mod_h2/issues/86
Moritz's user avatar
  • 238

Only top scored, non community-wiki answers of a minimum length are eligible