some one commented that ZFS is not a distibuted file system and thus more suitable for storage from a box than a cluster.

is it possible to start small as NAS with Nexenta / opensolaris and scale up in future by adding more SAS JBODs?

if so, what needs to be taken care of at the hardware list? for better performance?

i mean what to choose for storage adapters, HBA etc

  • 1
    It's better if you add your storage requirements in terms os IOPS and capacity. If you start designing a solution out of nothing, be prepared to have many iterations over hardware and software changes as requirements change. – Giovanni Tirloni Oct 6 '10 at 15:45

ZFS isn't a cluster file system, that's for sure, so can be used for single-server access but not for multiple. That said it makes a wonderful basis for a non-clustered NAS.

As for the Nexenta question - yes you can scale it very well from small to very large quite easily.

For performance I'd be tempted to stick to a single 4-6 core CPU, you won't benefit from many more cores. Try to get at least 4GB so you'll have a nice large cache. Use Dual teamed NICs. When picking controllers pick a name brand, you can't really go wrong with Adaptec, make sure it supports exposing all disks raw when in JBOD mode (i.e. if you have 16 drives does it appear as 16 drives, what's the most of these supported per controller). Always go SAS rather than SATA (although those new RE3 disks are quite nice and are genuine 24/365 drives too). Make sure you get dual power supplies. If possible use dual-port SAS disks and ensure both paths are live. Setup whatever alerting/monitoring you can with the controller/s to let you know IMMEDIATELY when you have a dead disk etc. Oh and buy at least one spare disk to be kept 'cold' and if you can have another set as 'hot' standby.

Other than that get on with it :)

  • hi, DL370G6 with Xeon 55xx, 16GB RAM + MDS 600 as JBOD. This JBOD has 46 15K drives...4 SAS ports. DL may have 4 1Gb NICs...do i need more for connecting 4 clients/consumers? can OSOL / Nexenta do hybrid provision? that is iscsi, NFS or CIFS from the same box? – John-ZFS Oct 6 '10 at 12:12
  • Go for an E5630 if you can, it's a nice balance of cost/heat/performance/price. Don't do 16GB of memory, those chips don't work like that, they go up in 3 chips per physcial CPU at a time (i.e. for a single CPU box get 3 x 1GB, 3 x 2GB, 3 x 4GB etc. not 2 x something or 4 x something), 12GB will be fine. HP support that box with Solaris so that's good, all the drivers should be there - might need to check the drivers for the disk controller as none of the P212/P411/P812 officially support Solaris - which model are you going for? 370 does have 4 NICs, nic teaming is supported too so you're ok – Chopper3 Oct 6 '10 at 12:28
  • thanks for your valuable feedback.P410 with 512 cache. anything to worry about SAS interface cards on MDS600? also thinking of HP procurve 3500yl Gb switch-layer3. overall is this a nicely balanced ZFS based solution? – John-ZFS Oct 6 '10 at 12:31
  • The P410 is an internal only card - you'll need an external one with the mini-SAS connectors to drive the 600. The 212/411 and 812 are the ones you're looking for. My only worry is the lack of official Solaris driver support for those three cards - might be work asking HP directly or doing a bit of googling. I know almost nothing of HP switches sorry, I'm a Cisco-man through and through when it comes to networking. Perhaps ask any questions about the HP as a different question? – Chopper3 Oct 6 '10 at 13:30
  • looks like HP P410 is based on PMC Sierra chipset. likely supported by opensolaris: opensolaris.org/jive/thread.jspa?messageID=458335 – John-ZFS Oct 9 '10 at 7:05

i actually build these kinds of systems for a living.

SuperMicro kit works great. Stay with Intel for NIC's. Use SSD's for ZIL (mirror this for gods sake) and L2ARC. Stripe RAIDz2 across the JBOD and the entire ARRAY to give you're self max safety. De-Dupe will need at least 24GB of RAM, use ECC DDR3 Registered and throw the money at this and a couple of 5660's if you can!

You can indeed start small - something like a http://www.supermicro.com/products/chassis/3U/?chs=836 (you will need to buy mobo,mem and proc for this).

Or take a look at http://www.va-technologies.com/sbb

  • i cant go with SuperMicro, we have an agreement with HP and i have chosen DL370 G6 + MDS600 for a NAS solution based on ZFS/Nexenta. The MDS600 has 4 SAS ports...DL370 G6 has 4 NIC ports do i need more? – John-ZFS Oct 6 '10 at 12:09
  • You connecting from the MSD6000 to the DL370 and then serving out from the 4NIC? 1GbE? Bond them for better throughput or go for 10GbE I would suggest. Make sure you have Intel NIC's as the broadcom support on OpenSolaris is - at the moment, it's been worked on - sub-par. – Khushil Oct 6 '10 at 14:47

Nexentastor actually now offers the Namespace Cluster plugin for use with their Enterprise server version. I don't have personal experience using this plugin, but it is supposed to allow you to horizontally scale additional Nexentastor servers into one cluster that acts as one filesystem via NFS. It looks like you migrate data between the server nodes without user interruption, and you can manage all the servers in the cluster from one node.

Here's Nexenta's product page on the plugin with more details: http://www.nexenta.com/corp/namespace-plugin

Using that plugin, you can not only add more JBOD enclosures to a head unit, but can also add more head units to load balance the workload across the cluster.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.