I have a weird issue with a server I am setting up. It's for a filesharing-type website, so fast IO and plenty of capacity are the requirements. OS is CentOS 6.4 64-bit
The server in question is a HP DL360p, with 18x drive bays populated with 2TB SAS drives in RAID50
There's also a HP StorageWorks SAS expansion bay with a further 12x2TB's, also in RAID50
RAID was configured using the server's BIOS configuration utility, the controllers used are pretty good ones with battery backup and 2GB FWBC.
Now, originally we set these up as separate volumes, but due to the specifics of our software, it would work out much better to have a single, large volume.
So, I set up an LVM volume combining these two volumes, then formatted the logical volume using XFS
The problem is, the resulting speed is disappointing. Running hdparm -tT gives a best read speed of 300MB/s
So I did a few tests and got this:
no LVM, XFS on both: Both volumes get around 700MB/s read speeds
with LVM, but volume not mounted: 1000-1100MB/s
with LVM in striped mode, volume not mounted: 1100-1300MB/s
So somehow XFS seems to be restricting the speed... I tried some more advanced options when formatting and mounting, such as -l internal, enabling lazy-count, nobarrier, but this yielded no improvement.
The only thing I found that may be a problem, the strip size of the RAID volumes did not match (one was set at 512KB and the other as 256KB), so I am re-configuring them to match, which will take a few hours more. I also reformatted the volume with su=512k,sw=28 (sw=28 is just a guess, as there are 28 active HDDs altogether... or should I set this to 2 for the RAID volumes?)
I'm tempted to just wipe out the whole thing and try ZFS, it seems promising, but I think configuring it would be far beyond my skill level...
So, if anyone has any experience or advice on this, it would be much appreciated!