I have experience with ZFS running on OpenSolaris on a 50TB NFS files server for HPC, so I will answer your questions based on that.
What is the CPU overhead of ZFS?
It's very small. It will vary depending on what checksum and compression algorithms you choose and it you enable deduplication. I have all 3 enables with the default options and rarely get my 16 cores to utilize more than 15% each. Keep in mind that compression and dedup also reduce the amount of data that you need to write so things actually endup happening faster at the expense of minor CPU utilization. CPUs are bloody fast nowadays.
From what I learned, when ZFS updates files, it keeps the old file
as snapshot, and write the updated
part for the new version. However
that would mean for each snapshot it
keeps that require significant storage
overhead. How much is this storage
overhead? For example, suppose I have
2TB usable space, how much space can
actually be used for the latest
version of files one year later?
The snapshots store only a difference of what's changed. You only start seeing the snapshots take up space if you delete or modify existing data. For mail server that stores mail in plain text that would mean that only the deleted emails will result in overhead in the snapshots. If you accumulate have 1.5GB (after ZFS compression) of emails and 0.5GB were delete over time then you will be able to fit everything into your 2GB zpool no matter how many snapshots you made.
Having 1 or more snapshots means that you will not be able to free up space by deleting files but you can free-up space by deleting snapshots.
ZFS is a transactional fs so even deleting a snapshot will require writing a small log to disk. This means that if your have 0 bytes of free space then you can't delete. I got stuck like that once. So take some care to setup a disk space quota (say 99% of your zpool) so that when you run out of space you will be able actually delete things.
Is FreeBSD with ZFS hosting virtualbox serving half dozen busy
guest mission critical mail servers a
reasonable combination? Anything
particular to be careful with? And
can I still choose ZFS for the guest
OSs? This is because I may build
another identical such box for
redundancy, and will need to do some
mirroring between each pair of the
guest systems across the boxes.
I don't know how well VirtualBox will work under this kind of load. You should test the performance before you deploy. Replicating would be the best with
I'm trying to configure a Dell R710 for this. From what I learned, I
shouldn't choose any RAID at all, is
that true? In that case, are the
drives still arrive hot swappable?
If you format the drives as JBODs then you can use ZFS's ZRAID. They will be hot swappable.
We have a SAN from LSI and we did not use ZRAID. Instead we relied on the hardware RAID6. There were cases when ZFS detected data corruption and I was able to tell which files were affected. The data was restored later by the hardware but if we had ZRAID the there would not be any visible data corruption at the file level.
If I need more space in
the future, I would just hot swap some
drivers with larger capacity to expand
the storage. There is no problem with
It's a good question. This would be a problem if you do hardware RAID. On the other hand, ZFS should be able to let you expand like that with ZRAID. I never tired that. When expanding we just add new shelves and create new zpools. Growing an existing zpools would be just as easy as adding new ones.