I've been reading everything I can find about partition alignment and filesystem optimization (ext4 and xfs) but still don't know enough to be confident in setting up my current configuration. My remaining confusion comes from the LVM layer and if I should use raid parameters on the filesystem in guest os'es.

My main questions are:

  1. When I use 'pvcreate --dataalignment' do I use the stripe-width as calculated for a filesystem on RAID (128kB for ext4 in my situation), the Stripe size of the RAID set (256kB), something else altogether, or do I not need this?

  2. When I create ext2/3/4 or xfs filesystems in guests on the Logical Volumes, should I add the settings for the underlying RAID (e.g. mkfs.ext4 -b 4096 -E stride=64,stripe-width=128)?

  3. Does anyone see any glaring errors in my set up below? I'm running some benchmarks now but haven't done enough to start comparing results.

I have four drives in RAID 10 on a 3ware 9750-4i controller (more details on the settings below) giving me a 6.0TB device at /dev/sda. Here is my partition table:

Model: LSI 9750-4i DISK (scsi)
Disk /dev/sda: 5722024MiB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start      End         Size        File system     Name      Flags
 1      1.00MiB    257MiB      256MiB      ext4            BOOTPART  boot
 2      257MiB     4353MiB     4096MiB     linux-swap(v1)
 3      4353MiB    266497MiB   262144MiB   ext4
 4      266497MiB  4460801MiB  4194304MiB

Partition 1 is to be the /boot partition for my xen host.
Partition 2 is swap.
Partition 3 is to be the root (/) for my xen host.
Partition 4 is to be (the only) physical volume to be used by LVM
(for those who are counting, I left about 1.2TB unallocated for now)

For my Xen guests, I usually create a Logical Volume of the needed size and present it to the guests for them to partition as needed. I know there are other ways of handling that but this method works best for my situation.

Here's the hardware of interest on my CentOS 6.3 Xen Host:

4x Seagate Barracuda 3TB ST3000DM001 Drives (sector size: 512 logical/4096 physical)

3ware 9750-4i w/BBU  (sector size reported: 512 logical/512 physical)
    All four drives make up a RAID 10 array.
    Stripe: 256kB
    Write Cache enabled
    Read Cache: intelligent
    StoreSave: Balance



Specifically in answer to your questions:

  1. Stripe width (256KB)

  2. Yes

  3. There are two things I can think of that aren't mentioned:

3.1) You aren't starting with the top level I/O size, which is where you need to start. If your application is mostly going to write in x KB blocks, then you need top optimise and align the entire storage stack for x KB blocks.

3.2) You haven't touched upon the block group alignment (-g option for ext* file systems). You need to adjust your block group size in a way that avoids all of the block groups starting on the same disk, thus creating a massive hot-spot and making that disk the bottleneck.

I wrote an article on this subject that explains how to achieve optimal storage stack alignment, which you may find useful.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.