2

I have a PC with a 256 GB M.2 SSD drive used for Linux Mint root, swap and home partitions right now. I have a 2TB HDD for all my data (I keep that separate from home dir actually, because users' home folder has user configuration).

I would like to migrate that system to ZFS to take advantage of checksums, snapshots, transparent compression and the power-loss robustness.

Since I didn't build the hardware with ZFS in mind I wonder now what would be the best way to utilize this setup.

I am doing lots of video production among other things, so I have big files (10GB) being thrown around a lot. I thought about using SSD as an SLOG, but wouldn't that unnecessarily wear the SSD off, when I'm capturing and rendering video files - HDD speed in not a bottleneck for me in this regard anyway.

Should I create separate pools for SSD and HDD? Can I easily move datasets between the pools with send/receive commands?

Could I partition the SSD to use it both for root partition and SLOG for HDD? Wouldn't that defeat wear leveling and kill my SSD faster? Wouldn't that kill SSDs performance benefits?

As a side-note:

I might add a second 2 TB HDD later as a mirror to gain redundancy. I also have 2 1TB USB 3.0 drives that I used with ZFS in Raid-0 for a while. They seem to work pretty well, and can handle ~130 MB/s write speed. I wonder if using that as a mirror vdev for my main HDD would be a good idea. They have proven to be stable for a few months (I know USB can be problematic with ZFS - I played with that a quite lot too).

I currently do rdiff-backup to an external 3TB USB 3.0 drive.

I have deployed ZFS a few times so far, once in a production environment for system root, and data storage in 2-disk mirror. I never used SSDs with it though.

What would you advise me to do?

1 Answer 1

2

I am doing lots of video production among other things, so I have big files (10GB) being thrown around a lot. I thought about using SSD as an SLOG, but wouldn't that unnecessarily wear the SSD off, when I'm capturing and rendering video files - HDD speed in not a bottleneck for me in this regard anyway.

Only sync writes go to the ZIL, so you may not see much difference in performance depending on the software. Write endurance is a concern only with low-quality consumer drives (you did not specify your model) or 24/7 heavy write workloads. If you work 8 hours a day, you have 3 times as much time left.

Should I create separate pools for SSD and HDD? Can I easily move datasets between the pools with send/receive commands?

Send/recv works without problems regardless of the underlying vdev configuration. Just keep in mind that you can choose each file system individually, but cannot get any finer granularity (file level).

Could I partition the SSD to use it both for root partition and SLOG for HDD? Wouldn't that defeat wear leveling and kill my SSD faster? Wouldn't that kill SSDs performance benefits?

Yes you could, 4 GB is plenty of space for the ZIL. It will on the other hand certainly be slower, but with fast SSDs, you may not notice this as much. Be aware however that you have now two problems in one - if your SSD dies, you will lose your whole operating system/config files AND may lose committed but not yet written data on the HDD. If you go this route, I strongly suggest using two SSDs, if possible two with same speed and latency, because the slower one sets the pace.

I might add a second 2 TB HDD later as a mirror to gain redundancy. I also have 2 1TB USB 3.0 drives that I used with ZFS in Raid-0 for a while. They seem to work pretty well, and can handle ~130 MB/s write speed. I wonder if using that as a mirror vdev for my main HDD would be a good idea. They have proven to be stable for a few months (I know USB can be problematic with ZFS - I played with that a quite lot too).

If it works for you and your other alternatives (for example putting the disk inside the system with SATA) are not available, it may be an alternative. Of course, regular backups are a must (as they are in any case).

I currently do rdiff-backup to an external 3TB USB 3.0 drive.

You could also utilize send/recv if you format the drive as a basic ZFS vdev. backup will be faster (block-level incremental sends) and you will have at least consistency checks (checksums and scrubbing). The only downside is that you put all your eggs in one basket regarding critical file system bugs, the choice is up to you here.

2
  • My home studio machine definitely gets much less than 8h/day uptime. I use all consumer-grade components. So I can do quick incremental send/recv using snapshots? Awesome! I would then still use my external 3TB for backup - but I'd use ZFS snapshots for that. I need to test this thoroughly to be sure I can backup and restore files from sent snapshots and remove old snapshots without braking all consecutive ones. You say that I put all my eggs in one basket - do you mean that in the context of using a single SSD for both storage and SLOG or what?
    – unfa
    Feb 23, 2018 at 13:28
  • @unfa In the context of using just one file system. If for example you have a second backup option, like cloud backup every week, or another hard disk with another file system like NTFS, you are resilient against any ZFS errors (not that there will be many more, but nobody can rule it out 100%).
    – user121391
    Feb 26, 2018 at 8:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .