4

I have very little experience with SANs, so please excuse this newbie.

One of our production sites has an HP EVA8000 SAN with a bunch of disks (forget exactly how much). it has been configured into a bunch of raid5's and the SQL server uses one RAID volume for data and another for the Log.

So far so good. Or so I thought

Today i ran some speed tests. Using rdfc i created a 100 MB file on the main data disk while the system was heavily used. This showed between 5 and 20 MB/s throughput for write, which, i thought was a bit on the low side. I then took down the application for maintenance, and ran the test again. This time on a quiet system I got 50MB/s write performance.

Is that not a bit on the low side? I mean, I get the same from the SATA drive in my desktop.

I then tried writing 500MB to one drive (the data drive), while writing 500MB to another drive (the log drive). Those two writes interferred with each other. Is that a sign of something wrongly setup?

I don't know what I had expected, but I'm currently reviewing the system to try and locate bottlenecks.

So, i guess my question would be. What kind of performance would you expect (read/write) from a (probably) expensive SAN using Fiber connectors?

(No, its not the little netgear thingy; its this) http://h10010.www1.hp.com/wwpc/ca/en/sm/WF05a/12135568-12135820-12135820-12208992-12208992-12236532.html

2
  • 1
    In one of your comments, you say that you were actually getting 50 Megabytes per second, which is normally written as 50MB/sec. You've consistently used Mb/s above, which is Megabits per second. Can you clarify what you're actually getting? Jun 22, 2009 at 22:22
  • 1
    Sorry, my bad. Have changed it now.
    – Soraz
    Jun 23, 2009 at 4:32

6 Answers 6

5

We have an EVA6100 so I've done tests on similar hardware. The 8000 is, IIRC, an older system than the 6100.

First off, your writes to the DB volume and Log volume are very likely contending for writes as it sounds like they're on the same Disk Group inside the EVA. A disk group is a grouping of disks that you create LUNs on, and each LUN is a disk presented to one (or more) servers. It is not at all uncommon to house many, many LUNs in a single disk-group. If you only have one disk-group, then the benefits of separating your log I/O from your DB I/O are somewhat reduced.

Second, if that DG is running into true I/O contention, then it'll degrade performance. This is something that's very hard to check from a consumer point of view, it's the kind of thing the SAN maintainers need to be aware of. Ask them if they know what the average controller CPU load is, as that'll significantly affect your write performance.

Third, know what your I/O mix really is an benchmark for that, not just file-copy speed. If your DB volume I/O is 80/20 read/write, then you really want to check read-path throughput. If it's the other way around, then your write performance is the metric you really need to check.

3

By simply writing a large file you do not measure storage performance. You just measure the performance of writing a large file. You are sadly mistaken if you think you will get anything similar when: writing randomly all over the disk, writing many blocks simultaneously, writing really large or really small blocks. I guarantee you, that you can have from your array max 1 MB/s on one type of workload, and then with another workload/setup you will easily get 800 MB/s (...provided your server can handle that much).

If you want to learn about these, use a proper tool. I've used Intel IOMeter at the time and I learned a lot with it. Probably now there are some better tools. The main parameter to watch out for is the queue size and random-vs-sequential operation. Read about disk mechanics, why the seek time is so unbelievably slow, and how RAID5 functions.

If you want to know how your storage is performing in real life, take a look at PhysicalDisk counters in perfmon.exe during your work day. Then create some load on your actual application and observe what happens.

3
  • Is IOMeter safe to use on the real disk, or should I get a new set of disks setup for this? My PhysicalDisk counters are all over the place. Average Disk Queue Length goes (visually in perfmon) from around 20 to 100 back down to zero (shortly). I would say it averaged perhaps 35. (I never did grasp how to read the scale part, so perhaps 35 is really 3,5, i dunno).
    – Soraz
    Jun 22, 2009 at 20:55
  • +1 for this. RAID5 is not the recommended storage for databases, especially not the logs.
    – pauska
    Jun 22, 2009 at 21:17
  • 1
    Soraz, IOMeter is not safe for production. First thing it does is the test file, and it tries to fill up your logical disk to 100% percent with it. Obviously not nice. You have to use restart trick to alleviate this.
    – kubanczyk
    Jun 22, 2009 at 21:41
3

Another important thing to check is for proper disk alignment. More than once, I have seen non-aligned LUNs be the source of big performance problems. If the offset on the partition isn't set before formatting, you can see 25% performance hits on IO.

RAID 10 is also a better option, at least for the log LUNs.

Finally, you can adjust your read/write cache ratio on the LUN to 100% write, since SQL Server does a great job of caching for reads.

SQLIO is a good tool for benchmarks, but definitely not for production. It is easy to use, runs sequential and random read/write tests of variable size, and makes for a good baseline to compare to other systems.

2
2

Look in to RAID10. A smart coworker once convinced me that it's better than RAID5 for write intensive databases, and he was right. It is.

1
  • Amen. The only argument against RAID 10 is reduced capacity, which IMO is a BS argument since platters grow at least 20%/year. Jun 23, 2009 at 3:34
1

It doe seem very low, EVA8xxx boxes are pretty quick when you consider they're built for ease of use. We have lots of them for various things and I have a spare 8100 with 112 * 1TB 7.2k disks and I can easily flood a 4Gbps FC connection, even on a RAID5 vDisk - that's about 385MBps.

I think something's up, I'd happily offer to help but it could take a while, you might be better getting HP in if you have a good enough relationship with them.

Let me know how you get on ok.

3
  • Thank you for the reply. You say 385MBps, is that Mega-bit-per-second? If so, does that mean my 50 Megabyte / second roughly matches that?
    – Soraz
    Jun 22, 2009 at 20:40
  • After noticing i write Mb/s instead of what i mean MB/s, i take it yours means 385Megabyte per second
    – Soraz
    Jun 23, 2009 at 7:10
  • 1
    I did, generally a upper-case/capital 'B' is bytes while a lower case 'b' means bits.
    – Chopper3
    Jun 23, 2009 at 7:41
0

What kind of disks are in the SAN (fibre channel, sata etc) , that can have a major effect on the speed your getting. Also when you are doing the tests i take it you are copying from the servers directly connected to the SAN?

1
  • Not really sure as to the types of disk, but I my test setup was "Create an XMB random file on the filesystem" from the server it was attached to, or mounted on (what is that called?). Will check types.
    – Soraz
    Jun 22, 2009 at 20:37

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .