I run a website which currently hosts approximately 300GB (over 1000 files) of training videos on a shared hosting provider. We increase that by ~50 video files per month (~20GB). Currently our backups have been on the desktop machines of our staff, however I'd like to set something up that is more automated. I will be looking into other hosting options, but in the meantime, I would like opinions/improvements regarding the following plan for backups on this server.
There are two types of files that will be backed up. The first is the video files described above. These only need to be backed up once per file, as they will never change. The second type to backup is files from the site itself. These should be backed up regularly and tracked for revisions. Most of the changes here will not be coding changes, and the staff making the changes are 1) not technically inclined and 2) distributed throughout the US. I don't think that an svn-based solution will work well given these facts.
So here's what I am thinking:
- Create a DB table to log backups. This table will include: hash of the file, modification date, size, date of backup, local path (at the time of backup), and path to the remote version of the file.
- Use a script running on a cron job to regularly (daily? weekly? monthly?) navigate the directory structure to identify files that have not been backed up. This identification can be done by comparing the hashes.
- After identifying the files that need to be transferred, the script will ftp them to the remote server. After each file is transferred successfully, a record of that transfer will be inserted to the DB.
Do you see any problems with this approach? Will I run into issues the first time the script executes, due to the large amount of data to be transferred during the first go-round?