0

Is there a tool that makes possible to backup a Mysql database to Amazon S3 or Amazon Glacier without having o create a local file with the database contents?

Something like that:

mysqldump -u root -ppass -h host --all-databases | magical-s3-tool s3-bucket backup-yyyy-mm-dd.sql

This magical tool would use the pipe data and transfer the backup data directly to S3, without creating a local file.

EDIT:

My goal is to avoid having to use a VM with a huge disk for running the backup process.

  • You could probably stream it into an S3FS mountpoint. – ceejayoz Dec 18 '12 at 20:15
  • 1
    Why are you trying to avoid the creation of a file? The answer to this might help frame the recommended approach. – Eric Hammond Dec 18 '12 at 20:48
  • I am not sure about S3 but you can often pipe over SSH like this: mysqldump | ssh remote-box "cat > file.sql" – jeffatrackaid Dec 18 '12 at 21:13
  • @Eric Hammond yes, i'm trying to avoid the creation of a file – Rubem Azenha Dec 18 '12 at 21:24
  • @RubemAzenha: Yes, that's clear... but why? (performance, security, regulations, size, disk space, for the fun of it, ...?) If you can explain what your underlying goals are, that can lead to a better answer that meets those goals. You can simply update the question above with your criteria, philosophies, and driving constraints. – Eric Hammond Dec 19 '12 at 22:58
1

You can use this:

mysqldump \
    --host=127.0.0.1 --port=1234 \
    --user='username' --password='password' \
    --single-transaction --routines --triggers --compress --compact \
    --databases '<database name>' | gzip | aws s3 cp - s3://'<bucket name>'/'<database name>'.sql.gz
  • thanks... Do you know how do the same thing with mydumper? I know it would be harder since mydumper has multiple files. – Rubem Azenha Apr 23 '18 at 19:18
  • I'd recommend using the default AWS CLI.. What is the use case of mydumper anyway ? – Varun Chandak Apr 24 '18 at 6:04
0

There is a tool https://github.com/vsespb/mt-aws-glacier which can do this (Amazon Glacier only). See --stdin option in documentation.

Note that this operation will block your mysqldump (if network is bottleneck), and this can cause problems with mysql lockings etc.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.