13

I'm planning to use ZFS on my system (HP ML370 G5, Smart Array P400, 8 SAS disk). I want ZFS to manage all disks individually, so it can utilize better scheduling (i.e. I want to use software RAID feature in ZFS).

The problem is, I can't find a way to disable RAID feature on the RAID controller. Right now, the controller aggregates all of the disks into one big RAID-5 volume. So ZFS can't see individual disk.

Is there any way to acomplish this setup?

  • 2
    Disable hardware RAID in favour of software RAID? That just doesn't make sense to me, especially as you have such a well proven controller. – John Gardeniers Jul 17 '09 at 4:49
  • 5
    Well, this is IMHO debatable. But I think ZFS has clear advantages over pure hardware RAID. – Arie K Jul 17 '09 at 6:35
  • 4
    @JohnGardeniers You obviously knew nothing of ZFS when you wrote that comment. If your lack of understanding has now been corrected, you may want to consider tidying up (deleting) your comment. – Mark Booth Mar 12 '14 at 11:02
  • @JohnGardeniers P400/P410i doesn't support device passthrough, i.e. TRIM/discard with RAID arrays will be unsupported, so, if you have SSD's, they might and die very soon in intensive applications. If application is not that intensive - don't use SSD's. – GioMac Jul 7 '16 at 19:20
-1

If you simply delete the volume in the RAID BIOS that exposes the disks directly in some versions of the SmartArray controllers. We always use the controller even with things like ZFS.

  • This is the right answer - just delete the array in the ACU, that way the BIOS just sees the disk. That said are you SURE you don't want it managed by the P400? they're a damn good card. – Chopper3 Jun 21 '09 at 14:46
  • Aha, so this bring back ol' software vs hardware RAID war :) Should I combine both setup: ZFS' pool on top of single RAID-5 (or even RAID 1+0) volume exposed by P400? – Arie K Jun 21 '09 at 16:18
  • 12
    There are advantages to letting ZFS have the raw disks over a hardware raid controller. For one if you have a disk go bad ZFS will only need to rebuild the actual data rather than the entire block device speeding up rebuild and reducing the possibility of running into another problem while rebuilding. By giving ZFS the entire raw disk ZFS also manages the drive cache which improves efficiency. Third ZFS will properly detect and correct IO errors due to its exhaustive data checksumming. Because of this I wouldn't put a hardware raid inbetween ZFS and the Disks. – 3dinfluence Jun 22 '09 at 3:41
  • 2
    Adding my thoughts here, since this came up in my search results... In my case, I'm using an HP Smart Array P212. It looks like this controller can not present the raw disks (no jbod mode). If you delete all raid volumes, it just doesn't present anything to the OS. I want to use linux software raid (md raid) because: 1) no raid 6 support on this controller (may be able to purchase a license to add this) which I want since I'm attaching 24 drives, and 2) I'm using desktop drives, which I hear md raid handles desktop drives better than enterprisey controllers that expect enterprise drives. – James Jan 3 '12 at 14:32
  • 3
    This is wrong. There's no way to present RAW drives from a Smart Array controller. – ewwhite Nov 4 '12 at 7:45
19

It is not possible to disable the RAID functionality of HP Smart Array controllers older than the P420i

A common "solution" to this problem is to create single-disk RAID-0 volumes at the controller level. This is not a good solution and definitely is not equivalent to a JBOD arrangement. There's RAID metadata on the disks, and the failures will produce unexpected results.

In this case, be careful with the Smart Array P400 single-disk RAID 0 setup. If you have a hotplug event (disk failure/drive removal), ZFS won't recognize the new disk without a reboot. There's no true JBOD setting on these cards.

Using a dedicated SAS HBA is the right path.

  • What is the difference between JBOD and creating RAID0 volumes for each disk? Are there performance penalties or certain risks? – Tommiie Jan 22 at 15:37
  • Less perfomance. More of a management thing. Failure of a disk == failure of the LUN or logical drive... so you may have to reconfigure or add a new logical drive to handle a disk replacement. – ewwhite Jan 22 at 19:10
9

It should also be noted, just because I just killed 7TB of data by following LapTop006's, well, lets call it "personal opinion", that a P400 Controller would expose unassigned disks as JBOD, that this is nothing but a guess, and it is false, at least for my P400. There may be other controllers behaving like LapTop006 said, the P400 does not, at least not with the original firmware (V2.75).

I learned this the hard way today when trying to bring over a 6 disc software RAID-5 from a machine with a faulty 6 channel SATA RAID controller. They had always been part of a software RAID, the RAID functionality of the ICH9 "Fake" RAID controller had never been used anyway.

The target machine didn't have enough SATA ports, so I thought, well, no problem, it is a SOFTWARE RAID anyway, why not attach the disks to a P400, the disks would - if the controller behaved like stated - appear as JBOD, and the OS would - like it had done many times before when I moved software RAIDs from one machine to another - recognize the RAID.

In my case, however, the P400 did recognize the disks as new and - without seeking my confirmation - it did auto-create a RAID-5 array at the controller level as soon as I powered up the computer. Bye-Bye software RAID.

I brought the disks back to the original machine, but the RAID had already been corrupted, the OS saw 6 empty disks now.

Bye, 7TB of data.

Damage already done, I played with the disks a little bit. Back at the P400 equipped machine, I deleted the unwanted RAID-5, the disks didn't appear at the OS level. I had to create 6 RAID-0 disks, and they appeared - all empty, however.

Conclusions:

  • The P400 does not pass unassigned disks to the OS.
  • You need to create RAID-0 configs to get the disks through to the OS.
  • Saving the RAID-0 config (or any other P400 config) will empty the disk(s).
  • The P400 auto-config may have killed anything on the disks anyway, by creating a RAID-5 without asking permission.
2

The other option is to create each disk as a single-disk RAID-0 array (yes, it should let you do this). Then ZFS will see all the indidual disks.

The 'technical' name for the option you're looking for is called JBOD - Just a Bunch of Disks.

  • 1
    Would this introduce additional overhead on the RAID controller? – Arie K Jun 21 '09 at 16:20
  • Don't know, I doubt it, but it looks like the other answer by Laptop006 is more appropriate as he seems to have more experience with this scenario. – Mark Henderson Jun 21 '09 at 21:24
2

Here is my personal experience with a HP Smart Array P410 RAID Controller:

I have a P410 controller bought off ebay. It came with 512MB of RAM, "Flash" backed, with "SuperCapacitor". I also found it came loaded with a "Feature License", which is called "Advanced Pack 01". This "Advanced Pack" license enables Raid 6 and 60, and other "Advanced Settings". Right now, I am not sure if this "Feature" is what get enabled when you add the RAM and Battery/SuperCapacitor to the card - I am thinking it is... please chime in, anybody, if you have some additional info.

I took a working 320GB SATA disk which I had loaded files onto, and plugged it into a port on the HP P410 controller. I then configured it with the HP "Smart Storage Administrator" (A Windows Utility) as a "Single Disk Raid 0" as others have commented.

So, after doing this, Windows immediately pops up the new drive. Lo and behold, my data is all there, nice!

You will notice, if you check the "Physical Device" info in Smart Storage Administrator there is an item in the list which reads: "Exposed To OS: No". Seems the P410 knows we want JBOD but refuses to give it to us! That's rather frustrating.....

Now, here's what I found. The Smart Array controller overwrites some data at the beginning of the disk and... if you try to take this drive out of the Smart Array and go back into a regular motherboard port, Windows Disk Management shows it as "Unallocated".

The solution to this..... Use the excellent "TestDisk" utility, you can select the physical disk, tell it you had an Intel partition, and after a quick search, it should find your missing information. You can use the "P: List Files" to se ethat those files are indeed there... Then you tell TestDisk to "Write" the info back to the disk, reboot the computer, and voila! Back to happy-land. So, there seems to be a way to get the job done here, you just have to be aware that the Smart Array (and many other controllers from what I've read) will over-write a bit of data on any disk you decide to set up as a RAID array of any sort. However, if you're careful and you don't initialize or erase, you can often get your drive back to working well by using a nice little tool such as "TestDisk".

Note: I have also had good success running a Raid 5 on the P410, Originally, I created it on a P400, then decided to swap it out for a P410. No going into the configuration menu or other intervention needed, the P410 recognized the Raid 5 array and immediately made it available, upon boot-up! I though that was quite nice. So, some more evidence that the HP Raid cards store all the array info on the drives themselves, not the card. Nice!

One other thing, I found no way to enable a true "Pass-Through" on this Smart Array. Leaving disks "Unconfigured" or otherwise not a member of an array prevented them from being shown to the O/S. The Smart Array will only play nice if you let it administer the raw disks. That having been said.... what I learned tells me the level of "interference" on the part of the Smart Array is minimal... to me it seems like not a big deal.

BTW I am screwing around with all this on a SuperMicro 745 Server/Workstation. The 743, 745, 747, and 748 are amazing chassis/systems! You can pick up a 743 or 745 fairly cheaply on ebay. And if you love MEGA storage, you can pick up an 846 (24 bays) cheaply on ebay (I got one).

Jeff

1

I simply moved the internal SCSI cable from the smart array card to the on board SCSI Controller located on the system board. This gave me access to the raw disks.

  • 3
    That may apply to older pre-G5 ProLiant servers. It's not valid on modern systems because they no longer have onboard SCSI. – ewwhite Mar 6 '12 at 19:01

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.