I'm planning on building out a file server using ZFS and BSD, and I was hoping to make it more expandable by attaching drives stored in other machines in the same rack via iSCSI (e.g., one machine is running ZFS, and others have iSCSI targets available to be connected to by the ZFS box and added to zpools).
Looking for other people who have tried this has pretty much lead me to resources about exposing iSCSI shares on top of ZFS, but nothing about the reverse. Primarily I have the following questions:
- Is iSCSI over gigabit ethernet fast enough for this purpose, or would I have to switch to 10GbE to get decent performance?
- What would happen when one of the machines running iSCSI targets disconnects from the network?
- Is there a better way to do this that I just am not clever enough to have realized?
Capacity wise, initially this would be about 10TB of data (not accounting for redundancy), and a reasonable goal for the foreseeable future would be scaling to 20TB, so with redundancy, probably about 40-50TB of total storage. If possible, we also want to keep all the data encrypted using GELI.
Thanks for any help.