Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We have a 6TB backup-to-disk system running with Backup Exec 2010. Each week, a full backup is carried out with differentials on the other days. We manage to get about four weeks of backups in there.

Firstly, am I right in thinking that the deduplication option would more effective use this backup-to-disk space? For example, in each of those four weeks, the same indentical 4GB file is backed up four times (space used = 16GB) but with deduplication only the first copy would be stored?

Secondly, if you enable de-duplication, does it have an immediate effect or does it take time for the deduplication to ripple through the B2D area?

The former would require a job to run on the existing B2D files, locating duplicates and marking up the BKF file as overwriteable.

The later would be far easier to implement as it just effects backups going forward.

If I were a betting man, I'd go for the later option ;-) Easier to code...

share|improve this question

2 Answers 2

The normal setup for using B2D against deduplicated storage systems (or BE's deduplication engine) is to run 1 full backup, and then "incremental forever". This is the preffered method to utilize the deduplication to it's full potential, but it might not suit every datacenter out there.

Deduplication comes in many flavours. I can't remember right now wich one BE uses, but they all create checksums of data blocks and then compare it to a database to see if it's already been stored somewhere else.

Backup Exec have quite the hefty system requirements for running deduplication, you should be aware of this. If I recall correctly it's 1GB of RAM per 1TB of data in the backup cycle.

You should notice the deduplication effect after the full backup has run with the option enabled. This will basicaly be your "base" data as described above, where every incremental backup will deduplicate against the full backup.

I don't see any need to use the existing B2D files. Why not just point your next full backup against a folder on the B2D storage called "dedupe" or something like that?

share|improve this answer
    
That is a large RAM requirement! I've not managed to find out yet at what level BE does de-duplication either - bit, byte or file. I'm just provisioning a virtual W2k3 server so I can install the trial version of BE 2010 to see if I can gleam more from the program itself. –  Rob Nicholson Sep 17 '10 at 15:22
up vote 2 down vote accepted

I've now installed a trial version of Backup Exec 2010 and can answer my own questions:

Firstly, you cannot use an existing backup-to-disk system and convert it to deduplication storage. Deduplication is a completely different mechanism and is a new type of storage in BE. You create a new deduplication storage area much like you create a backup-to-disk storage area.

Therefore my question about whether it de-dupes an existing B2D folder is "no".

This does give one the thorny problem of switching over to de-dup from B2D if you are trying to use the same storage media. As BE never deletes B2D BKF files, you'll have to manually do this as the media expires.

Secondly, BE de-dup is block based with 64k blocks by default. On top of this is the dictionary database is has to maintain to allow it to hash duplicate blocks. The de-dup folder structure is much more complex than B2D.

Thirdly, yes BE 2010 has a big RAM requirement. I was testing in a 1GB W2k3 VM and noticed that it ran like a three legged donkey. It was consuming 1.5GB in total so was thrashing the page file. I therefore think we need to upgrade our live backup server before we consider using de-dup.

Cheers, Rob.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.