3

Hey StackOverlow Community,

I currently am using some software that stores a bunch of XML files. All the files are under 25k bytes. The makers of the software state the performance is based on multiple concurrent tiny reads. The faster you can read from the drives the better it can perform.

After, loads of performance testing they seem to be telling the truth. However, before throwing money at this I want to test as much as possible. The server doesn't ever seem to get past 35% CPU usage or ever peak memory wise.

Server setup is as follows

  • Windows Server 2003 Standard
  • Quad Core 2.6 Ghz
  • 4GB Ram
  • 2 Hardware Raid Cards
  • 8 15k SCSI drives in RAID10 (hardware raid)

I am thinking if I change the setup to use 2 Intel X-25 M in RAID0 I would get better performance. Backups aren't an issue. We have a live replication to another server and an offsite nightly backup which also goes to tape. I currently tried a single Intel X-25 on a different server and it seems promising.

So the questions I have are as follows -

  1. Should I expect better performance from 2 SSD drive setup than the 8 SCSI drive performance?
  2. Any special SSD setup steps that I should take? (Block size, etc?)
  3. Currently testing in Windows Server 2003? Does this hinder in anyway? Should I move to Windows Server 2008? (Linux, etc. aren't valid options)
  4. Any tweaks to the OS that I should do?
  5. Different/Better benchmark software?

Anyone experience or test anything similar please let me know.

Some benchmark info using (CrystalDiskMark) 5 passes 50Mb

SSD Box

Type - Read Mb/s - Write Mb/s  
Seq  - 225       - 50.84  
512K - 152.7     - 53.73  
4K   - 18.85     - 35.35  

SCSI Box

Type - Read Mb/s - Write Mb/s  
Seq  - 549.8     - 352.2  
512K - 538.3     - 342.2  
4K   - 89.30     - 65.19

TL;DR - Are 2 SSD drives in RAID0 or RAID10 faster than 8 15k SCSI drives in RAID10?

Thanks for reading.

7
  • Helvick's advice seems good. I just want to point out that you can't do RAID 10 with only two SSDs. You need four drives to do RAID 10. I wonder how many SSDs it would take to beat the SCSI drives. See my next comment.
    – pplrppl
    Nov 4, 2009 at 22:42
  • You did not mention specifics of the SSDs tested. Variants that would affect the math are X-25e, X-25m G1, X25m G2 80GB, X25m G2 160GB Each of these have a different transfer rates and some handle fragmenation/long term usage issues better than others.
    – pplrppl
    Nov 4, 2009 at 22:46
  • Edited to specify X25 M. In theory if its possible I would use 8 drives to mimic the already existing raid10 setup.
    – Demetri
    Nov 5, 2009 at 15:44
  • Which X25 M? Is it G1 or G2? Is it 80GB or 160GB? Between those four combinations there are three performance profiles.
    – pplrppl
    Nov 5, 2009 at 15:59
  • 80 GB. How can I find out G1 or G2? The drive is newegg.com/Product/Product.aspx?Item=N82E16820167016
    – Demetri
    Nov 5, 2009 at 16:31

4 Answers 4

3

Given the performance levels you are trying to hit then you should bear a couple of things in mind.

The earlier suggestion to format the drive holding your data set with an NTFS cluster size > 25k is a good one, 32k seems like a good choice for your use case. The purpose of doing this is to ensure you're avoiding have to deal with fragmentation if at all possible, and lowering the file system overhead associated with reading in a single file.

Also I'd suggest taking a look at your RAID stripe size. If the nature of your data set results in (mostly) sequential IO then a larger stripe size is more beneficial, if it's random then a smaller stripe size will be smarter, just don't make it any smaller than your files system cluster size. Given the description of what you're trying to do I'd say your IO profile is mostly random so a stripe size of 64k would be fine but it might be worth experimenting with.

What's very important is to make sure the partition is aligned - you will want to use Diskpart.exe on your systems, generally setting an alignment offset of 64k does the trick with standard cluster and RAID stripe sizes but a 1024k offset is used in Vista\Windows7\W2K8 as it will ensure alignment even with larger stripe & cluster sizes. There is a very good article on SQL server performance from Microsoft here that explains the reasons why this is important to do for high performance drives\arrays. The short version of this is that a poorly aligned partition can degrade IO performance by 15-30%.

For SSD's the same general rules apply but the underlying behavior of reads\writes is vastly different. Rather than dealing with 512byte disk sectors at the most fundamental level IO on SSD's uses much larger blocks. Reads tend to be in fixed page sizes of between 4k and 128k, writes involve buffering and large erase block sizes (in the megabyte range). The key thing for you (where read IO is important) is that you want to set your RAID stripe size to be a multiple of the read page size for the disk type you select (I think the Intel X-25's all use a 128k read block) and you want to set your alignment offset to some number that ensures it is greater than that. The standard recommendation of a 64k partition offset would be a poor choice if the SSD has a 128k read page size for example. Because of the asymmetric nature of SSD writes it isn't easy to optimize SSD RAID arrays for write performance but that's a whole 'nother story and involves lots of cache.

There's a nice article on optimizing SSD RAID here on the OCZForum, it's aimed at enthusiast setups but they get the general message right for anyone trying to roll their own SSD RAID from off the shelf kit as far as I can tell.

Finally if your read IO pattern is predominantly sequential the 8 15k drives can (in theory) hit somewhere in the ballpark of 800Meg/sec if not more. Two SSD's, even Intel X-25E's, will only hit about half that. I'd guess that your read IO pattern is biased enough towards random IO to negate this but under the right circumstances your 8 15k SCSI drives can be a lot faster than the two SSD's. This is borne out by your testing. Looking at those numbers though I'd say that some work with partition alignment and stripe sizes will help quite a bit.

2
  • So if I am understanding this correctly, 2 SSD in RAID0 setup with the appropriate block size, page size and partition offset should be slower than the 8 15k SCSI drives. However, 8 SSD drives setup in RAID 10 should be faster. Any guesses on how much faster?
    – Demetri
    Nov 5, 2009 at 15:46
  • 2
    Everything depends on the IO profile - as the bias tends towards sequential rather than random IO then 8 x 15k SCSI drives will out perform 2 x SSD's. The problem with 8 SSDs in RAID 10 is getting a controller that can keep up but if you can find one then for 100% sequential IO the SSDs would top out around 25-40% higher transfer rates. For 100% random IO a single X-25 class SSD should be about >5x faster than your RAID 10 iSCSI pack. In reality it depends and if there is any writing going on at all it will seriously disrupt all the assumptions made above.
    – Helvick
    Nov 5, 2009 at 16:00
2

If the XML files are all smaller than 25k, consider formatting your drive with a block size > 25k. The logic being that an entire file will fit into a single block and therefore you will not get fragmentation across individual files. A single read/write operation would then cover an entire file.

This would probably be pretty poor and a waste of space for your O/S drive, so I'd make sure you are not doing this on the same partition that your operating system is installed on. Leave that on the default block size (probably 4k NTFS as you're using windows)

This is more of a longer term performance improvement as I doubt you are going to have much fragmentation on a fresh install, hence you might not have noticed minor performance degradation with age - depends on your setup.

2
  • 1
    +1 for the block size. The bottleneck is the disk response time, so give it every advantage you can. Nov 4, 2009 at 21:30
  • @John, thanks for comment and you're right on the disk response time - probably seek time is going to be killer for spindle disks. A decent raid controller should keep the disks spinning. Possibly one could get away with cheaper raid controller on the SSD drives. Nov 4, 2009 at 23:38
0

I think you should seriously look into loading those XML files into the RAM. Since you're on Windows, there are plenty of tools that can create a RAM drive from your actual RAM, just keep in mind that it all goes poof when you reboot, so you need to also have those files on the disk. Buying more RAM is going to be pretty cheap, though depending on your current use I think you'll be fine just using the 4GB you have.

1
  • I was thinking about the RAM option. One issue is that the top level directory that stores all these files is currently around 35 GB. This would mean new server with a lot of ram.
    – Demetri
    Nov 4, 2009 at 20:14
0

I'd first recommend disabling last access time updating of files/directories with your setup. With that many files it will have noticeable impact.

If only concurrent reads are an issue you can try to set this 8 disks to RAID-1 only (instead of RAID-10) as this would allow for a read of any file from any drive, which would be idle at time of reading. You'll lose capacity though (only 1 drive's worth will be available, not an issue for under 100GB of data) and writes will be slower (any write would have to be done to all drives).

Adding the maximum possible amount of RAM to a server and migrating the OS to 64bit (to avoid PAE) will also be nice, because of a bigger file cache.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .