I'm trying to figure out why the performance of a Virtual drive built from 12 SSDs is not really faster than a single SSD drive.

The Server is a IBM x3650 M4, two Xeon E5-2643v2 130W 6C 3,5GHz. The integrated RAID Controller ServerRAID M5110e has a BBU and 1GB Flash Option. Connected are 12 SSDs "IBM SSD S3500 240GB SATA 6,4cm MLC HS, 6.0 GB/s".

The RAID1 or RAID10 Virtual drives were created with the MegaRAID Storage Manager with the following properties: no read ahead, Write Back with BBU, Direct IO, 128k stripe size.

I tested the Performance with ATTO Disk Benchmark and here are the results of 64k read/writes (for other block size read/writes the Performance does not scale also):

2 drives in RAID1: Write 1969 MB/s Read 2458 MB/s
4 drives in RAID1: Write 1801 MB/s Read 2365 MB/s
6 drives in RAID1: Write 1614 MB/s Read 2381 MB/s
12 drives in RAID1: Write 1544 MB/s Read 2432 MB/s
12 drives in RAID10: Write 1738 MB/s Read 2465 MB/s 

I've also ordered the IBM Option

"MegaRAID FastPath SSD performance acceleration MegaRAID FastPath software provides high-performance I/O acceleration for SSD-based virtual drives by exploiting an extremely low latency I/O path to increase the maximum I/O per second (IOPS) capability of the controller. This feature boosts the performance of applications with a highly random data storage access pattern, such as transactional databases. The feature is activated by enabling M5100 Series Performance Accelerator (90Y4273)."

But from what I understand this boosts the Performance of a Virtual Driver only by the factor of 2 or 3.

I'm wondering if the Controller is a bottleneck ?

I tested now with IOmeter with a 20 GB dataset, one worker, 64 KiB 100% Read, 0% random, running for five minutes:

RAID1 of 2 drives: Total IOPS 6917, Total MBPS 453
RAID1 of 4 drives: Total IOPS 9121, Total MBPS 597
RAID1 of 6 drives: Total IOPS 11186, Total MBPS 733
RAID1 of 8 drives: Total IOPS 12959, Total MBPS 850
RAID1 of 10 drives: Total IOPS 14677, Total MBPS 962
RAID1 of 12 drives: Total IOPS 16351, Total MBPS 1071 
RAID10 of 12 drives: Total IOPS 18390, Total MBPS 1205
RAID0 of 6 drives: Total IOPS 16140, Total MBPS 1057
RAID0 of 12 drives: Total IOPS 19562, Total MBPS 1282
RAID5 of 12 drives: Total IOPS 18988, Total MBPS 1244

Tests with 50% reads, 50% writes:
RAID5 of 12 drives: Total IOPS 4621, Total MBPS 303
RAID10 of 12 drives: Total IOPS 4967, Total MBPS 325
RAID10FP of 12 drives: Total IOPS 11803, Total MBPS 773 

RAID10FP is the measurement when the purchased FastPath option is activated.

  • The controller definitely is a bottleneck... The problem is, the more drives you use, the faster the read access in RAID1 and RAID10 should become but in fact most of the controllers struggle with keeping the drives in sync resulting in the advantage of many drives being eaten up by the computing power needed to sync the drives. – Broco Aug 13 '14 at 20:49
  • 1
    Are you sure your benchmarking tool is measuring what you want? How much RAM is in the server? How big is the data set that the ATTO tool is processing? And what units are your results in? Millions of megabytes? – ewwhite Aug 13 '14 at 21:07
  • Something looks wrong there. Is that MegaBytes per second? and you say it doesn't scale with larger block sizes? something is seriously wrong. My ethernet network is faster. You shouldn't be getting such slow speeds from SSD drives no matter the configuration. Check the system logs and make sure there are no errors that need to be addressed. – Matt Aug 13 '14 at 21:27
  • 1
    I think your benchmarking tool is broken, and it's actually reading/writing to/from cache rather than the disks. – Michael Hampton Aug 13 '14 at 21:47
  • 1
    Can you test with iometer? – ewwhite Aug 13 '14 at 21:52

Your iometer results look to be consistent with what you were expecting. I think that the testing tool/protocol was the issue here, and that there's nothing wrong with your array. Be sure to use FastPath, though, as it really does improve SSD array performance...

| improve this answer | |

The problem here is the write back feature of LSI controller, which only make sense for HDD´s. For SSD´s always use write through and no read ahead and DirectIO (even with RAID5). Then you see the expected performance with more SSD´s...

| improve this answer | |

Your card isn't able to send data fast enough to saturate the drives. That said, while SSD are fast all around, they're screaming fast on random seek reads. Try running a benchmark using small block random reads and measure the number of IO/s you can get.

| improve this answer | |
  • I tested this with IOmeter also, 4k 100%read, 100% random, 2 workers with 64 outstanding I/Os per target. With a RAID1 of 12 drives I get 204567 IOPS and 837 Total MBs/Second. – Peter Sawatzki Aug 15 '14 at 6:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.