up vote 1 down vote favorite
share [g+] share [fb]

I'm in a evaluation phase of some hw that could be used for setting up a hadoop cluster. This hw is refurbished (hp G6 servers w/ Smart Array 410/i controller) and probably we should/must use it... we haven't it yet. I've read that 410/i controller doesn't allow a JBOD configuration as required by hadoop, is there anybody that could confirm this statement ? Or not ? Seems that a workaround could be to configure raid0 on every single disk, again anybody confirms it or there is other ways of configuration ? Thank you Rob

link|improve this question
feedback

3 Answers

up vote 1 down vote

While the Pxxx range doesn't allow direct JBOD configuration if you just create one array off all your disks and then create multiple RAID0 arrays of the equivalent sizes then you're pretty much there in terms of functional similarity. That said I'm also a little dubious of Hadoop's requirement, firstly there's no way it can actually tell what the real disk configuration as it's hidden by the BIOS and secondly because it seems overly prescriptive given how well thought of the HP controllers are, what's the problem with the array managing its disks?

link|improve this answer
1  
Thanks for the answer, HDFS (Hadoop file system) has built-in redundancy by replicating blocks across multiple nodes. RAID 0 is slower than the JBOD configuration used by HDFS because I/O is limited by the speed of the slowest disk in the RAID array I/O on JBOD is independent, so the average speed is greater than that of the slowest disk. Test done by Yahoo showed JBOD performing better than RAID 0: markmail.org/message/xmzc45zi25htr7ry – nysalsa May 10 '11 at 6:29
feedback
up vote 1 down vote

The Smart Array 410/i controller only allows 2 raid devices to be created without the optional memory card added to it. So if your hardware only has 2 drives, then it would be fine; you can just make it 2 raid0's. If you have more then 2 drives you will probably have to purchase Part #462974-001 - 256MB Battery Backed Write Cache BBWC memory module. If you aren't sure, give it a try in the configuration utility, it will complain that there is a limit on the number of configured raid devices allowed if you need to purchase the memory addon.

I recently setup a ZFS server utilizing one of those and 12 disks in RAID0 mode for each and it performs fine. Since you can disable pretty much all features of the actual raid controller, its effectively the same as JBOD mode.

link|improve this answer
While you can create multiple RAID 0 on that controller to be used with ZFS, you will have a problem if a disk fails. ZFS won't recognize the replacement unless you reboot. – ewwhite May 9 '11 at 19:51
Thanks for the answers – nysalsa May 10 '11 at 7:00
feedback
up vote 0 down vote

This situation is similar to what I encounter when designing hardware for ZFS storage solutions. ZFS is a software RAID and leverages the server's resources for disk management. Hardware RAID isn't recommended, and JBOD isn't really possible with Smart Array controllers. So when using ProLiant systems for ZFS, I replace the Smart Array controller with something that has a JBOD mode or a basic SAS/SATA HBA.

See the recommendations at: ZFS SAS/SATA controller recommendations

link|improve this answer
Thanks for the answer – nysalsa May 10 '11 at 7:00
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.