Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Usually after dumping a MySQL database with mysqldump command I immediately tar/gzip the resultant file. I'm looking for a way to do this in one command:

So from this:

mysqldump dbname -u root -p > dbname.sql
tar czvf dbname.sql.tgz dbname.sql
rm dbname.sql

To something like this:

mysqldump dbname -u root -p > some wizardry > dbname.sql.tgz

Or even better (since I'm usually scp'ing the dump file to another server):

mysqldump dbname -u root -p > send dbname.sql.tgz to user@host

I'm running bash on debian.

share|improve this question
add comment

7 Answers

up vote 56 down vote accepted
mysqldump --opt <database> | gzip -c | ssh user@wherever 'cat > /tmp/yourfile.sql.gz'

You can't use tar in a pipe like this, and you don't need it anyway, as you're only outputting a single file. tar is only useful if you have multiple files.

share|improve this answer
5  
You're right about not needing tar, but you could use it in the pipeline if you did, with mysqldump | tar cf - | gzip -c | ssh ... 'cat > file.tgz' –  Darren Chamberlain Jan 27 '10 at 0:15
    
Does that actually work? I'm pretty sure tar needs a list of filenames to work on. –  James Jan 27 '10 at 10:25
1  
I updated this to work locally (not on a remote ssh server) oh, and I use a dynamic name based on date, thanks to original poster & answerer! mysqldump --opt <database> | gzip -c | cat > $(date +%Y-%m-%d-%H.%M.%S).sql.gz –  electblake Apr 11 '11 at 13:41
3  
@electblake: you don't need to be using 'cat' if it's local. Just gzip -c > $(date +%Y-%m-%d-%H.%M.%S).sql.gz –  James Jan 3 '12 at 18:58
    
Just for fun, you could use netcat instead of piping to ssh. You wouold save a little on the encryption overhead of ssh, if it is being transfered over a secure network (or you don't care about security). Nowadays you might also consider using xz instead of gzip. –  James Nov 5 '12 at 2:10
add comment

Use a named pipe.

mkfifo mysql_pipe
gzip -9 -c < mysql_pipe > name_of_dump.gz &
mysqldump database > mysql_pipe 
rm mysql_pipe

I use it all the time, it'a awesome.

http://en.wikipedia.org/wiki/Named_pipe

share|improve this answer
4  
James does the same thing in 1 line. –  Jonathan Haddad Feb 3 '10 at 17:28
11  
..but learning about named pipes is worth it :-) –  Tomasz Zieliński Feb 15 '11 at 15:34
    
mkfifo mysql_pipe; gzip -9 -c < mysql_pipe > name_of_dump.gz &; mysqldump database > mysql_pipe; rm mysql_pipe there, one line. Of course I would keep the pipe around and use it every time. –  d34dh0r53 Jul 21 '12 at 7:20
add comment

I wrote a quick script to suck down a remote mysql database. It uses mysql compression, gzip and ssh compression. Sucked down a multi GB database at an incredible rate.

    ssh -C user@host "mysqldump --opt --compress database <table> | gzip -9 -c" > outputfile.sql.gz

A side benefit is that it requires no free space on the source database server, so you can use it to backup a database on a server with zero free disk space before going in an pruning your data.

Hope it helps somebody.

share|improve this answer
    
I've created a simple shell script: #!/bin/bash if [ -z "$1" ]; then echo "Usage: ${0} [host] [user] [database] [outputFile]" exit else HOST=$1 fi if [ -z "$2" ]; then echo "Usage: ${0} ${1} [user] [database] [outputFile]" exit else USER=$2 fi if [ -z "$3" ]; then echo "Usage: ${0} ${1} ${2} [database] [outputFile]" exit else DB=$3 fi if [ -z "$4" ]; then OUTFILE="${DB}.sql.gz" else OUTFILE=$4 fi COMMAND="ssh -C ${USER}@${HOST} \"mysqldump --opt ${DB} | gzip -9 -c\" > ${OUTFILE}" ssh -C ${USER}@${HOST} "mysqldump --opt ${DB} | gzip -9 -c" > ${OUTFILE} –  Tony Dillon Sep 12 '11 at 18:52
    
Helped me.. thanks. –  Syed Aslam Dec 28 '12 at 15:19
add comment

If you are running this locally just use the following command to backup your database & zip it using gzip:

mysqldump -u userName -p (passwordPrompt) yourDatabaseName | gzip -v > output.gz
share|improve this answer
1  
Yes, this is the most simple solution. I use it too. –  Roman Snitko Aug 9 '12 at 13:53
    
Simple and easy to use! Thanks! –  rxt Mar 27 '13 at 9:42
add comment

You can do like:

mysqldump --add-drop-table -h dbhost -u dbuser -p dbname (tablename tablename ... ) | gzip -c > wp.sql.gz

e.g.

mysqldump --add-drop-table -h localhost -u root -p wordpress | gzip -c > wp.sql.gz

share|improve this answer
add comment

Use pv and monitor rate!

mysqldump prod_db -h dbslave | pv | gzip -c > prod_2012_08_20.dump.tgz

Or, if you know the size (3GB), get an accurate estimate:

mysqldump prod_db -h dbslave | pv -s 3g | gzip -c > prod_2012_08_20.dump.tgz
share|improve this answer
add comment

I've been working on this bash script below that tries to put together all the good advices I've seen when it comes to dump/restore with mysql. It is targeted at remote operations.

Just reconfig vars and give it a try. :)

Features are:

  • you can pass a list of tables to dump (selective dump)
  • you can be prompted for passwords (MySQL/SSH) or set them in variables
  • network transmission is gzipped
  • you can opt to save gzipped dump to remote server
  • you can reimport dump to remote server on-the-fly (no temp files on local/remote server)
  • you have visual feedback of what is happening (thanks to echo and pv)
  • you can set mysql variables before and after the dump process

What needs improvement:

  • you need to pass a list of tables (cant dump all tables)
  • MySQL password are the same for source and target
  • you need to GRANT PRIVILEGES manually (looks like MySQL dont let do it remotelly)
  • you need to have installed sshpass
  • some innodb huge compressed tables are slow to dump (may be mysqldump's fault)

I share this script here hoping it can be improved by the community. (best viewed with nano or other editor that colors the code)

--------------------------------- cut here ----------------------------------

#!/bin/bash
#set -x

#REQUIRED VARS
SOURCE_USER=root   #MySQL user
SOURCE_HOST=localhost
SOURCE_PASSWORD=yourmysqlpass  #optional
SOURCE_DBNAME=yourdbname
TARGET_HOST=192.168.1.2
TARGET_DBNAME=yourdbname
TARGET_SSHUSER=root
TARGET_SSHPASSWORD=yoursshpass  #optional
TABLES='table1 table2 table3 table4'
TARGET_DIR="/data/dumpfiles"
EXEC_ACTION_TEXT[0]='Reimport TABLES directly into remote MySQL database'
EXEC_ACTION_TEXT[1]='Backup gzipped data to TARGED_DIR on remote TARGET_HOST'
EXEC_ACTION=0

#print config
echo "---------------------------------"
echo " SOURCE_USER:    $SOURCE_USER (MySQL)"
if [ "SOURCE_PASSWORD" != "" ]; then
echo " SOURCE_PASSWORD:<present>        "; else
echo " SOURCE_PASSWORD:<to be asked>    "
fi
echo " SOURCE_HOST:    $SOURCE_HOST     "
echo " SOURCE_DBNAME:  $SOURCE_DBNAME   "
echo " TARGET_HOST:    $TARGET_HOST     "
echo " TARGET_DBNAME:  $TARGET_DBNAME   "
echo " TARGET_SSHUSER: $TARGET_SSHUSER  "
if [ "TARGET_SSHPASSWORD" != "" ]; then
echo " TARGET_SSHPASS: <present>     "; else
echo " TARGET_SSHPASS: <to be asked>    "
fi
echo " TABLES:         $TABLES          "
echo " EXEC_ACTION:    $EXEC_ACTION - ${EXEC_ACTION_TEXT[$EXEC_ACTION]}"
echo " TARGET_DIR:     $TARGET_DIR (only for action 1)"
echo "---------------------------------"
echo "PRESS <ENTER> to continue...";  read;  echo

#read the mysql password from command-line (SOURCE and TARGET uses the same password)
if [ "$SOURCE_PASSWORD" == "" ]; then
     echo -n "Type $SOURCE_USER password for MySQL servers: "; read -s SOURCE_PASSWORD; echo
fi
echo "Creating database $TARGET_DBNAME on $TARGET_HOST if not exists ... "
mysql \
--user=$SOURCE_USER \
--password=$SOURCE_PASSWORD \
--host=$TARGET_HOST \
--execute "create database if not exists $TARGET_DBNAME;"

echo '--------------------------------------------------------------------------------------'
echo "**** ATTENTION ****: execute this command on mysql server at  $TARGET_HOST :"
echo "GRANT ALL PRIVILEGES ON $TARGET_DBNAME.* TO '$SOURCE_USER'@'%' IDENTIFIED BY 'yourpass';"
echo '--------------------------------------------------------------------------------------'
echo "PRESS <ENTER> to continue...";  read;  echo

#read the password from command-line
if [ "$TARGET_SSHPASSWORD" == "" ]; then
     echo -n "Type the password for remote SSH Server (TARGET) ['$TARGET_SSHUSER'@'$TARGET_HOST']: "; read -s TARGET_SSHPASSWORD; echo
fi

for thistable in $TABLES
do
     case "$EXEC_ACTION" in
         0)
         thisaction="gunzip | mysql --user=$SOURCE_USER --password=$SOURCE_PASSWORD -D $TARGET_DBNAME"
         endmessage='remote reimporting has finished'
         ;;
         1)
         thisaction="cat > $TARGET_DIR/`date +%Y.%m.%d`-"$thistable".gz"
         endmessage="$thisaction has finished"
         ;;
         *)   echo "EXEC_ACTION=$EXEC_ACTION not supported" && exit 1
     esac

     echo "---------------------------------------------------------------------"
     echo "-- table $thistable"
     echo "---------------------------------------------------------------------"
     (
       echo -n "-- setting variables... " > /dev/stderr  #talk to user via stderr
       echo "SET AUTOCOMMIT=0; SET UNIQUE_CHECKS=0; SET FOREIGN_KEY_CHECKS=0;"
       echo -n "starting mysqldump... " > /dev/stderr
       mysqldump --opt --user=$SOURCE_USER --password=$SOURCE_PASSWORD --host=$SOURCE_HOST $SOURCE_DBNAME $thistable
       echo -n "done mysqldump, reseting variables... " > /dev/stderr
       echo "SET FOREIGN_KEY_CHECKS=1; SET UNIQUE_CHECKS=1; SET AUTOCOMMIT=1;"
       echo -n "commiting... " > /dev/stderr
       echo "COMMIT;"
       echo "done!" > /dev/stderr
     ) | \
     gzip -c -2 | \
     pv | \
     sshpass -p $TARGET_SSHPASSWORD ssh $TARGET_SSHUSER'@'$TARGET_HOST $thisaction
     echo $endmessage ' with exit status '$?
done
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.