3TB + 4TB HDD - should I deploy ZFS?

Question

I have a production machine at work that was running on a single 3TB HDD. Now I've added another 4TB drive into it. The 3TB drive has been used for 1,5 years at this point, the 4TB is brand new.

I have about 1500 GB of unique data (ignoring backups) on the 43 TB drive at this point. I'll be storing about 500 GB of archives from other machines on the network, giving the Samba shares for accessing their old files. It doesn't need to be fast, but should be redundant (I have the only copy of the data).

I'm not sure how I should organize the storage to maximize security of the data with what I have right now. I will probably be able to add another 4 TB drive to this machine in 2 years, it'd be good to be ready for that.

I basically need 3 different types of storage:

1. System - place to boot and run my production Linux OS
2. Data - place to store current projects and files served to other users on the network
3. Archive - place to keep old projects (preferably with compression)
4. Backup for system and current data, should I blow anything up by accident at any time on my main pool.
5. Hopefully some redundancy for archive as well (as these will be only copies of old projects)

I'm thinking about partitioning my 4TB disk: 1 TB for SWAP, GRUB and Linux OS 3 TB for ZFS And adding 3TB mirror across the whole 3TB drive for redundancy.

ZFS snapshots could to some degree replace my usual production files backups, that'd save me a lot of space probably. However - if anything happens to the Zpool, I am in big trouble.

I know I can create single disk vdev (on a 3TB partition on the 4TB drive) I can then move there all my data, freeing up my 3TB drive. I could prepare a boot and swap partition (not sure how to do this though right now though) in the last 1TB of my 4TB disk, the clear my 3TB drive, and add it to my zpool as mirror, gaining redundancy.

Does this sound like a good idea?

I have about 12 years experience using Linux, I have built my own Kernel and installed Gentoo, I use it daily for all my work with graphic design and media production, I do a bit of system administration at the company where I work, but I would by no means call myself as sysadmin. I don't have a lot of ZFS experience - I mostly deployed it for fun on USB flash drives and USB hard disks in the past few months, trying to learn about it as much as I can.

An alternative is to just use the 4TB drive as an archive + backup storage with EXT4 + SWAP (to reduce the stress on the main drive - the SWAP is particularly been grinding hard at times for the past 1,5 years).

That could be a better idea, as it's way simpler and doesn't require me to move all my system to a new drive, and I still have redundant copies (though no filesystem compression or checksumming). Also I know the 3TB drive will probably die first so I'd rather move everything to the new 4TB one to put less stress o n the old 3TB one, so it can last longer.

Answer 1

There are a couple of ways you could go about using the second 4TB drive. One possible way is what you mentioned, setup the new 4TB drive and then copy the data. However I suggest this other approach.

• Copy all your data from the 3TB to the 4TB
• Setup the original 3TB as boot with ZFS
• Copy the data from the 4TB back to the 3TB (now with ZFS)
• Setup the 4TB as mirror of the 3TB. You would lose 1TB of usage from the 4TB, but later when you get another 4TB you can grow the vdev

PROs:

• You have your data in a safe state at every step of the operation. To be safe suggest you use rsync with checksum flag every time you are copying the data around.
• You end up with RAID 1 at the end

CONs:

• You lose 1TB from the 4TB. You may be able to install ZFS in a 3TB partition, but that would limit your ability to grow ZFS later
• It will take several rounds of copying your data around so may be a bit of time (mostly machine time though)

If the data is really important I would also suggest to use a backup to either a 3rd drive, if you have one, or to a cloud service like B2. You mentioned 1500GB. Looking at B2 pricing looks like it would be about $7.5 for that amount of data. Once you have finished the operation and you are on your end state (RAID 1 on ZFS) you could delete the backup on B2 if you wanted (although would likely be a good idea to just keep backing up). https://www.backblaze.com/b2/cloud-storage-pricing.html

Through this exercise the most important thing to try to make sure is that your data is safe. Right now you only have it on one drive so long term doing nothing is not good either.

Going with ZFS gains you snapshots and RAID1 which can be very useful in comparison to say putting the second drive in and just rsyncing to it or using it as extra storage. Snapshots are a great space saver when it comes to doing backups; specially if a lot of your data doesn't change.