I'm looking for a simple on-site (i.e. not online) backup solution for our small company. Right now we have approximately 4TB of data in total, maybe adding ~500GB per year. The amount of data changing per day is much less tough - I guess much less than 1GB on average.
All data is only accessed from the intranet and most machines are running Windows and some are running under MacOS if it matters.
The data in detail:
(a) Large parts of the data are images/videos/documentation(pdf) and alike, I would guess 2.5TB.
(b) Frequently accessed are our CAD-data files, but they only take up 10-20GB. These are controlled/accessed by a centralized CAD vcs called GAIN (I think it keeps it's data in a binary database). Currently this is dumped on the evening and then backed up.
(c) Some primarily source-code data are already under version control (SVN, GIT) taking up less than 2GB.
(d) Some programs only have binary source codes and are "archived" as zip-files. New versions are added and some old versions are restored sometimes but old versions never get changed. These programs take up roughly 80GB.
(e) Some personal backups (emails etc.) and other stuff takes roughly 1TB I guess.
(f) We also have a small amount of data on a single Microsoft SQL server. This should make up less than 1GB.
Right now we do a full backup every Monday to Friday evening from network disks to local server disk to tape drive on the server. We alternate the Friday tape, i.e. we have tapes labelled mo,tue,wed,thu,fri1,fri2. This implies we cannot go back in time more than 2 weeks in the worst case.
What's a good solution to this heterogeneous system consisting of
(a) large seldom accessed, seldom changed, seldom added data,
(b) frequent accessed rather small data delivered by a program internally using a database,
(c) frequent accessed rather small data under "common" version control,
(d) large binary files (~100MB) which are mostly added, seldom read, never changed (should optionally be disposable) and
(e) miscellaneous data like office files, data logs, mail folders which are seldom added/changed
(f) data on Microsoft SQL server
I'm firm with programming, version control and computers in general but new to backup strategies. So it would be good if the solution is quite simple to maintain.
If possible a versioning like that offered by SVN/Git would be nice, so the last successful backup allows restoration of every single file ever backed up (and not manually removed).
Problems with the strategy so far:
backing up takes a long time (15 hours)
=> There's not enough time to test the backup
=> Hard to tell if the backup is really working
=> What to do if the backup time reaches 24 hours?
- restoring a backup is quite a pain
- restoring something I deleted/modified/overwrote a month ago is not possible
A solution should address all these problems.
Time usage in detail:
Gathering of data from other servers over network to backup server: 02:15
Copy data on backup-server (which acts as a "regular" server as well) to another drive on the backup server: 09:00
Copy all data from internal drive on backup server to tape attached to backup server: 03:45