This is my first time building a machine with a hardware RAID card. We bought a Dell T620 with the
H710P RAID controller (1GB NV Cache), a
160GB Solid State Drive (SATA Read Intensive MLC 3Gbps), and two
3TB 7.2K RPM Near-Line SAS 6Gbps Hard Drives. The solid state drive is pretty much dedicated to the OS to keep it "hoppin".
The two SAS drives are configured by RAID 0. We are treating this space as scratch for analyses so we're not concerned about data loss. What we want is high-performance IO because we deal with lots of large files. For example, my current project is to work with 800 files ranging in size of 100-200GB. Unfortunately I have to transfer the files to the computer, analyze them, and delete them. Surprisingly (to me), is that I had 8 jobs running simultaneously (transferring, analyzing, deleting), and each job was on track for 20+ hour run times (compared to ~3 hours for a single job). I found that the processors were waiting on IO, according to top (time waiting for I/O completion was hovering around 20).
I realize that these are only 7.2k RPM drives, but I assumed they're pretty capable since Dell listed them at 6Gbps. BeowulfNode42 mentioned here that some drives get a 6Gbps interface for advertising, even though they can't even saturate a 3Gbps link. But I assume Dell wouldn't do that with a high-end server.
I strolled around google land to see if my expectations were unreasonable, but I didn't find anything definitive.
Question: What is a reasonable expectation for this setup? IO is obviously the bottle neck. The RAID card seems pretty nice and I thought the drives were pretty nice.
I ran hdparm to see what I'm getting. Here are the results:
>sudo /usr/sbin/hdparm -Tt /dev/sdb /dev/sdb: Timing cached reads: 19542 MB in 2.00 seconds = 9778.47 MB/sec Timing buffered disk reads: 1028 MB in 3.00 seconds = 342.11 MB/sec
The cached reads are pretty rockin', but I expected more from buffered reads. I believe the theoretical output for two 6Gbps drives is 750MB/s, so I expected to get somewhere around 600MB/s.
I appreciate your help. Other relevant information listed below. Please let me know if I missed anything.
OS: opensuse 13.1 RAM: 256GB (1866) CPUs: Dual Intel Xeon E5-2650v2 2.6GHz, 20M Cache, 8.0GT/s QPI