4

We recently acquired two "decked out" DL360's, complete with the Smart Array P400i RAID raid controller, 512MB BBWC, and (6) 10k SAS drives.

They are configured in a RAID 1+0 set up, running LVM and ext3.

Using the HP utilities (hpacucli), I set up the disk write cache (to be on), and the accelerator cache ratio to be 25% write, and 75% read.

I've also updated them to the latest firmware using the HP firmware disk (version 7.22 of the RAID firmware).

In looking around the web, I found this link: http://www.overclock.net/hard-drives-storage/515068-tuning-hp-smart-array-p400-linux.html

Which shows that with the read-ahead set properly, one can achieve some fairly amazing performance.

My own tests (same iozone performance) shows:

Initial write  191148.89    191.15
      Rewrite  204751.79    204.75
         Read  127655.03    127.66
      Re-read  126286.11    126.29
  Random read  148323.94    148.32
 Random write  185377.55    185.38

Most of my read numbers are 1/2 to 1/3 of the numbers being reported by (8) SAS drives, with xfs.

What am I missing? Does xfs and (2) drives make that much of a difference? Or, did I not configure something?

Any, lastly ... would software RAID be faster?

6
  • What generation (G#)? G6, G7 etc?
    – gWaldo
    Sep 7, 2011 at 1:39
  • He's referring to a DL360 G5, as the controller available with that model is the SmartArray P400i.
    – ewwhite
    Sep 7, 2011 at 1:58
  • 1
    Can you post your iozone command line?
    – ewwhite
    Sep 7, 2011 at 2:24
  • G5. The iozone is the exact same as the link I referenced.
    – Anthony
    Sep 7, 2011 at 3:12
  • @Neil - firstly this is a question and answer site, don't add a comment as a question, secondly he states in his first line that he's got '512MB BBWC'
    – Chopper3
    Sep 7, 2011 at 10:54

2 Answers 2

4

This is several questions in one, so I'll try to address a few of them.

I typically set Smart Array controllers to leverage a higher write cache ratio. I prefer to have 75% write cache because the OS (using the XFS filesystem) caches aggressively. XFS will make a difference, but what are you tuning for? Are you tuning to simply achieve specific numbers, or is there an application requirement driving this?

ext3 isn't the fastest filesystem out there. But you have some mount options (e.g. noatime) and journal settings you could tweak.
I don't use LVM, especially with HP controllers that can provide many of the same benefits.
You have I/O scheduler and elevator settings (e.g. noop or deadline, in this case) that can be tuned, but that's a function of your application's actual needs.

If you do go with XFS, try a basic config then try some advanced configuration settings. Over time, I've ended up with mount parameters very similar to the one in the original link.

I just ran the following iozone command line on an XFS partition contained within a DL380 G5 with P400i, 12GB RAM and 8 x 146GB 10k drives. The elevator is set to deadline:

Command line used: iozone -t1 -i0 -i1 -i2 -r1m -s24g

initial writers  =  348957.75 KB/sec
rewriters        =  335130.03 KB/sec
readers          =  132851.70 KB/sec
re-readers       =  137116.27 KB/sec
random readers   =   35774.41 KB/sec
random writers   =  250618.38 KB/sec
2
  • Awesome, thanks! I'm tuning for a 90% read / SELECT database, so reads are more important than writes (we might write 50MB per day). I had set the noop scheduler. I will try again with write 75%, and will also re-try using xfs and/or ext4.
    – Anthony
    Sep 7, 2011 at 3:12
  • On the iozone, you may want your -s parameter to be twice your RAM size. Otherwise, you may be skewing your random read results.
    – ewwhite
    Sep 7, 2011 at 3:22
1

ext3 is slow. If you know how to tune XFS, use that. Otherwise you should use ext4.

I'm personally a big fan of using hardware RAID unless you need your array set up in a way that the RAID controller doesn't support. (Others will likely disagree.)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .