I am looking for a glorified cron manager with a web interface to manage backups and uploading to cloud storage. (and yes webmin has a pretty good cron scheduler plugin)
In the old days enterprise backup systems were behemoth monstrosities costing 6 or more figure dollar prices or roll your own low level tools that are tightly coupled to their tape archiving forebears.
Examples of the former are HP data protector and IBM Tape storage systems, and of the latter I am thinking of Bacula and amanda.
These were great systems for their time, and did much of the grunt work for you ie.
- managing media
- and providing a very old school GUI (I think I remember Hp data protector had an ActiveX component, who uses activeX?)
However recent changes are away from TAPE and local storage and towards cloud or network based backup storage locations.
In addition, other trends such as automated provisioning has meant that there is far less file and directory backup "stuff"* and much more database and binary object backups objects such as SQL databases and SVN, or other repositories.
Another observation is that It costs money to leave data in the cloud
hence it is necessary to have some easy to use management interface to archive old backups on S3 or openstack swift networks.
So I end up in the situation where most of the backup management tools I am familiar with are not suitable for my backup purpose and now most of my apps need some sort of database backup dump, and the uploading the backed up files to network or cloud storage needs to be managed as well as the jobs.
For mySQL backups, I have ended up with a stack of software which is something like this;
1. percona xtrabackup to generate full and incremental
2. cron to initiate the backup jobs
3. duplicity with python-cloudfiles to sync the backup jobs to cloud
4. nagios and logwatch to monitor and alert for failures.
However this all seems a bit crufty.
The duplicity/cloudfiles solution is remarkably easy to configure with rsync like semantics, and xtra seems to do mostly hot backups without a hitch, but none of this really inspires the confidence that the One-stop-shop software like HP data protector, or tivoli did.
What I would like to do is tie all this together using some generalized job scheduling manager that could run the jobs, report on the success, and could handle scheduling of tasks for ageing old backups.
I am aware of some Java job scheduling tools like Quartz, but that is really too low level for my requirement, really I am thinking of some sort of glorified cron with reporting.
mysql-ZRM is interesting, it supports scheduling and basic reporting, and also there is an xtrabackup plug-in. however it really lacks a web interface and is probably not quite up to the percona level as it apparently not really maintained any more.
xtrabckup manager is good, supports scheduling. However it only supports native xtrabackup incrementals, which are massively large if you have lots of myisam tables.
holland and zmanada are a couple of derivative or commercial projects I am keeping an eye on.
duplicity and python-cloudfiles is quite the b*ll*cks at the moment, so that is what I am using to sync the backup files to a openstack swift storage.