Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

What things should I check to confirm that compression is actually happening on our tape backup system? Although the tapes are marked as 200G/520G (native/compressed) capacity, they seem to fill up before the 200G mark (some less than 100G).

I'm using

  • Sony AIT-4 tape autochanger
  • Sony SDX4-200C (AIT-4) tapes
  • Ubuntu Lucid
  • Bacula

I've tried checking hardware compression with: tapeinfo -f /dev/nst0, which gives

Product Type: Tape Drive
Vendor ID: 'SONY    '
Product ID: 'SDX-900V        '
Revision: '0102'
Attached Changer API: No
SerialNumber: '0001000036'
MinBlock: 2
MaxBlock: 8388608
SCSI ID: 1
SCSI LUN: 0
Ready: yes
BufferedMode: yes
Medium Type: Not Loaded
Density Code: 0x33
BlockSize: 0
DataCompEnabled: yes
DataCompCapable: yes
DataDeCompEnabled: yes
CompType: 0x3
DeCompType: 0x3
BOP: yes
Block Position: 0
Partition 0 Remaining Kbytes: 201778000
Partition 0 Size in Kbytes: 201779000
ActivePartition: 0
EarlyWarningSize: 0
NumPartitions: 0
MaxPartitions: 0

... so I presume it's on.

Notes:

  • The Bacula documentation says hardware compression needs to be enable with "system tools such as mt"
  • The data "should" compress well. The largest proportion is text files with lots of redundancy (genomic data). bzip often reduces to 10-30%
  • I have compression (i.e. software compression) turned off in Bacula
share|improve this question
1  
How are you determining that your tapes are "filling up before the 200G mark"? –  voretaq7 Sep 6 '12 at 21:13
    
In the Bacula GUI, under media. The tapes are marked eg 79G, but "Full". It's not a job limit, because the tape only has one job written to it –  pufferfish Sep 7 '12 at 8:54
    
If your tapes are filling up before even getting close to the uncompressed capacity you may be appending to the tapes and only seeing what has been added, rather than the full amount on them. –  John Gardeniers Sep 7 '12 at 10:20
    
Do you have UseVolumeOnce = yes or a low Maximum Volume Jobs = ### setting? in your Bacula configuration? Bacula's tape usage counter should be for the whole volume, but if you aren't seeing even the uncompressed capacity of your tapes it sounds like Bacula is misconfigured somehow. (You should however perform the tests Mircea and I outlined below to confirm that the problem is not the drive itself) –  voretaq7 Sep 7 '12 at 15:10

3 Answers 3

How old are your tapes? Our LTO-1 tapes often fill before the 100GB because they're OLD (new tapes fill at 140-150GB).

share|improve this answer

As Mircea pointed out the DataCompEnabled: yes in your tapeinfo output means that hardware compression is enabled for your tape drive.

Compression is not a magic thing that takes the input and reduces it in size by 50% which is what most tape vendors claim in their advertising -- If your Bacula installation is already doing gzip (or some other) compression your tape drive's hardware compression may not be optimally effective (in fact if Bacula is doing effective compression your tape drive's hardware compression may make the data take up more space).

If you want to test your tape drive's compression to be sure it's working properly the best way to go about it is usually to use tar to write a tape that consists of highly compressible data (text files are good for this) and see how much tape you use to store that data.

share|improve this answer

DataCompEnabled: yes means that the compression is enabled.

share|improve this answer
    
Yes, that's why I'm puzzled –  pufferfish Sep 6 '12 at 13:01
    
May be your data is not compressible. Try to write text files or HTMLs on the tape. –  Mircea Vutcovici Sep 6 '12 at 13:03
2  
Try disabling compression and see what happens, whether you get the exact 200GB you expect (and I presume that this isn't 1024 gigs vs 1000 gigs that is throwing you off?). –  Dan Pritts Sep 6 '12 at 14:21
    
Most of the large data files are text files with lots of redundancy. Compress well with for example gzip. –  pufferfish Sep 7 '12 at 8:56

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.