Tag Info

Hot answers tagged

67

mysqldump --opt <database> | gzip -c | ssh user@wherever 'cat > /tmp/yourfile.sql.gz' You can't use tar in a pipe like this, and you don't need it anyway, as you're only outputting a single file. tar is only useful if you have multiple files.


26

I tested this under nginx 1.3.9 with two files, and these were the results I got for the various levels: text/html - phpinfo(): 0 55.38 KiB (100.00% of original size) 1 11.22 KiB ( 20.26% of original size) 2 10.89 KiB ( 19.66% of original size) 3 10.60 KiB ( 19.14% of original size) 4 10.17 KiB ( 18.36% of original size) 5 9.79 KiB ( ...


25

You might have a look at zgrep. >$ zgrep -h grep through gzip files usage: zgrep [grep_options] pattern [files]


21

pigz is a parallel version of gzip. Although it only uses a single thread for decompression, it starts 3 additional threads for reading, writing, and check calculation. Your results may vary but we have seen significant improvement in decompression of some of our data sets. Once you install pigz the tar file can be extracted with: pigz -dc target.tar.gz ...


16

The zgrep program is available for Linux (and perhaps some Unix too). This will decompress the files and then grep through them.


13

Use a named pipe. mkfifo mysql_pipe gzip -9 -c < mysql_pipe > name_of_dump.gz & mysqldump database > mysql_pipe rm mysql_pipe I use it all the time, it'a awesome. http://en.wikipedia.org/wiki/Named_pipe


11

You can read about it in exorbitant detail in the HTTP specification, but here's the gist: when the browser needs to request a file, it first checks its local cache. There are three main possibilities: The browser has a local (cached) version of the file that is marked as expiring as a certain time, and that time is in the future. In this case, the browser ...


10

The zgrep program exists for this specific purpose. http://linux.about.com/library/cmd/blcmdl1_zgrep.htm


9

This small script seems to be your best option, given your requirements: cd directory for dir in */ do base=$(basename "$dir") tar -czf "${base}.tar.gz" "$dir" done It properly handles directories with spaces in their names.


9

See the Apache mod_deflate documentation, specifically, the "compress everything except images" example. It has worked well for me and would be put into an .htaccess file as follows: <IfModule mod_deflate.c> # Insert filter SetOutputFilter DEFLATE # Netscape 4.x has some problems... BrowserMatch ^Mozilla/4 ...


9

The first thing I'd try is running a LiveCD environment and just attempt to ungzip everything, hoping that would return the system to a bootable state. Note: I'd be concerned about potential data corruption if the original gzip process was interrupted. Otherwise I'd try to migrate the DB to a new system as others have suggested but as you've encountered ...


8

I wrote a quick script to suck down a remote mysql database. It uses mysql compression, gzip and ssh compression. Sucked down a multi GB database at an incredible rate. ssh -C user@host "mysqldump --opt --compress database <table> | gzip -9 -c" > outputfile.sql.gz A side benefit is that it requires no free space on the source database ...


8

Addition to Martin Fjordvalds answer: Apache uses chunked encoding only if the compressed file size is larger than the DeflateBufferSize. Increasing this buffer size will therefore prevent the server using chunked encoding also for larger files, causing the Content-Length to be sent even for zipped data. More Information is available here: ...


7

Also, by default IIS 6.0 is only configured to compress the following file types: htm, html, txt, asp, dll, exe. So if you're browsing to some other type of file (php or aspx, for example) it won't be compressed. You can configure additional file types to be compressed using the following commands: for static file types... cscript ...


7

myfile_`date '+%F_%T'`.sql.gz


7

Here's a backup/maintenance script I use: #!/usr/bin/sh #backup all mysql databases # list MySQL databases and dump each DIR= DATESTAMP=$(date +%Y%m%d) DB_USER= DB_PASS= # remove old backups find ${DIR} -type f -mtime +5 -exec rm -rf {} \; DB_LIST=`mysql -u $DB_USER -p"$DB_PASS" -e'show databases;'` DB_LIST=${DB_LIST##Database} for DB in $DB_LIST; do ...


7

Two questions: 1) What did I do wrong with the syntax? Gzip only compresses individual files; it's not an archiving tool. It's usually used in combination with something like tar. In fact, some versions of tar will use gzip to automatically create a compressed archive if given appropriate flags. For example: tar -cvz -f public_html.tar.gz ...


7

The level of gzip compression simply determines how compressed the data is on a scale from 1-9, where 9 is the most compressed. The trade-off is that the most compressed data usually requires the most work to compress/decompress, so if you have it set fairly high on a high-volume website, you may feel its effect. It sounds like your issues are more related ...


7

You have to enable the GzipFilter to make Jetty return compressed content. Have a look here on how to do that: http://blog.max.berger.name/2010/01/jetty-7-gzip-filter.html You can also use the gzip init parameter to make Jetty search for compressed content. That means if the file file.txt is requested, Jetty will watch for a file named file.txt.gz and ...


7

You probably want to use zip and not gzip. This should do it: zip -r newzip.zip /path/to/zip/stuff


7

What is the question you are asking exactly? “No space left on device” Would indicate that you've run out of disk space. Change the current working directory to another directory mounted on a larger partition and try again ... Also, bear in mind that du won't always match df. It is possible for files to be deleted but the handles still remain in use ...


7

/etc/logrotate.d/nginx /var/log/nginx/access_log { rotate 7 size 5k dateext dateformat -%Y-%m-%d missingok compress sharedscripts postrotate test -r /var/run/nginx.pid && kill -USR1 `cat /var/run/nginx.pid` endscript }


7

This will depend on whether the filesystems are repaired enough for you to be able to mount those partitions from a LiveCD. Don't bother trying to boot the system yet. First, mount the partitions and unzip all the .gz files. This will give you working copies of init and system binaries. Then you can use grub to repair the boot sector. Then boot to single ...


7

Is there some parameter that allows my tar file to be resumed after some problem? Nope. What you should do, though, is run your command from within a terminal multiplexer like screen or tmux. That way if your connection drops, the process keeps running.


6

If possible, use PNG as image format. It's the defacto standard for lossless image compression and will probably surpass the compression level achieved by merely gziping bitmaps. If you need to use BMP because of dependencies from legacy systems, you can try using the output compression methods available in your webserver, e.g. see mod_deflate for Apache. ...


6

The key issue is "how much data is being compressed?". If you are running a massive DB query that takes a noticeable number of seconds to run, and the resulting page is a few tens of Kb long, then the expense of compressing the data will be completely dwarfed by the expense of the SQL work to the point where there is no point even thinking about it. A ...


6

Debian/Ubuntu have the a2* set of scripts for managing modules and vhosts. You can use a2enmod: $ sudo a2enmod deflate Enabling module deflate. Run '/etc/init.d/apache2 restart' to activate new configuration! $ sudo /etc/init.d/apache2 restart * Restarting web server apache2 ... waiting [ OK ] $ ...


6

If you're risking overwriting files in production with uncertain results, you're doing something wrong. Test first. tar zxvf archive.tar.gz path/to/file/in/archive -C /destination/dir


6

I enabled deflate on static assets on my site (by MIME type) using the following added to the .htaccess file at the root of my public_html directory: <ifModule mod_deflate.c> AddOutputFilterByType DEFLATE text/html text/xml text/css text/plain AddOutputFilterByType DEFLATE text/javascript application/javascript application/x-javascript ...


6

HTML5 Boilerplate (http://html5boilerplate.com) offers what seems to be the best and most effective solution setting along with plenty of others like caching, mime types etc. Highly recommended. <IfModule mod_deflate.c> # Force compression for mangled headers. # http://developer.yahoo.com/blogs/ydn/posts/2010/12/pushing-beyond-gzipping <IfModule ...



Only top voted, non community-wiki answers of a minimum length are eligible