We are in the process of moving our (direct attached) storage systems to an iSCSI based consolidate storage and we have come up with this interesting question: where should we place LVM processing? Let me explain.

We are configuring a consolidated storage system that is based on the three level methodology ie storage, filer, application server. That essentially separates storage devices from file-systems. The storage is based (among others) by DELL systems running CentOS providing iSCSI devices to the filers. The filers "consume" iSCSI storage by providing NFS/CIFS/etc file-systems to the application servers. Storage devices provide also iSCSI devices to the virtual hosts (KVM) that directly make use of it in the form of VGs for their virtual machines.

Here is the question? Where do we place LVM processing? In other words should we partition (ie create PP/VG/LV with LVM on the storage servers or on the Filers/VMhosts?

Our initial approach was to postpone as much of the processing/translation of storage (LVM and file-systems) to upper level hosts (filers/application servers) since there will be several of those and only few storage servers. In this way we distribute processing more effectively. Are we right? Are we grossly doing something wrong? thanks.

  • 1
    Are you building your own storage NAS/SAN solution, or are you using something commercial? – ewwhite Mar 21 '12 at 12:50
  • We are actually doing both - NetAPP FAS2040 and DELL R510's. My question actually concerns the R510's that run the CentOS OS. – ank Mar 21 '12 at 12:54

You gain the most flexibility by using LVM on the iSCSI servers themselves, and then again on the filers. This allows you to resize the iSCSI devices you're presenting (iSCSI-server LVM) and then resize the volumes on the filer to accommodate (filer LVM). It does reduce performance a very small amount, but it really increases your ability to handle odd cases. In my judgment the trade-off is a good one.

You can do the multi-layer LVM thing since the iSCSI-server LVM is invisible to the filers (it's just a bunch of blocks in a LV), and when the filer drops LVM datastructures on the storage the iSCS-server won't see it unless you do some weird things.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.