I have a setup in which one SAN storage connected by FibreChannel is accessible by two hosts (at block level). And I want to create two lvm2 physical volumes on the storage: pv1 and pv2, for now being the only members of the two volume groups: vg1 and vg2 respectively. My hosts should use them exclusively and "don't touch the other".

Since SAN is visible by both hosts, the lvm structure will be visible by both. (I tested it: creating logical volumes on one host results in them being visible to the other under /dev/mapper). I want to make sure that host1 will only modify vg1 and host2 will only modify vg2 to prevent data corruption.

Is it guaranteed that when I don't explicitly execute any lvm commands, lvm daemons and kernel won't reshuffle extents/optimise the volume group belonging to the other host, thereby corrupting it?

  • What is your goal in doing this? Can you create two different LUNs? – Law29 Aug 24 '16 at 22:04
  • I don't have access to LUN configuration. – Met Aug 24 '16 at 22:06
  • The original goal of the setup was to have a proper parallel filesystem (i.e. gfs2). But I gave up after two days of trying to implement it. Moreover it seems that the performance of gfs2 is bad. So I want to set up beeGFS, but it is SAN-agnostic and designed for storage attached directly to hosts. So in my setup I want to separate the storage into two chunks. – Met Aug 24 '16 at 22:09
  • Of course I could simply create two regular partitions instead of lvm, but I am planning to add more storage in the future and would like to easily manage it. – Met Aug 24 '16 at 22:14
  • OK. If you want to be safe, create partitions and use them as PVs (I'm not sure how you are creating two PVs out of one LUN to begin with). – Law29 Aug 24 '16 at 22:42

No, it won't corrupt anything just by having the VGs visible on both systems, but having the VGs visible on both hosts is dangerous (due to the likelihood of some process, script, or human messing up and picking the wrong disk to operate on). You can filter them out, so that when performing PV scans, each host ignores the LUN that it shouldn't be touching. Thusly, each will only detect one VG at a time. This is useful in your scenario, and many others.

As an aside (and before I delve into this), you might consider handling this at the SAN level and separating these LUNs into separate targets. That way the base block devices never even get picked up by the "wrong" initiators in the first place. I would recommend this option over others unless there is a really good reason you want the LUNs visible on both hosts simultaneously.

Onward, we're looking at creating filters within the LVM configuration. This is controlled via a series of regular expressions to restrict "pvscan" from detecting certain block devices (or classes of block devices). The file that we make this change in is /etc/lvm/lvm.conf

Within that config, you will find (about half way down) a line that states "global_filter=". The file should give some hints on how to use this, but let's go over some concepts.

First, you should be allowing and restricting block devices based on paths that are not subject to change. DO NOT use things like /dev/sda to select a disk. That can change, and your filters wouldn't be reliable. Instead, use something like /dev/disk/by-id/ or /dev/disk/by-path/. Both of these directories contain symbolic links that use actual information about your LUNs to identify them. by-id uses the SCSI ID, and by-path uses the physical host path and bus type. Use by-id when possible, as it's the most reliable. Because these are symbolic links in the aforementioned directories, you can use `ls -l' to match them up with the /dev/sd* paths they correspond to.

Using that by-id path, you can make a global LVM filter to make scans reject that undesirable block device in future scans. When constructing this line, put allow entries first (denoted by "a"), and reject entries afterwards (denoted by "r"). An example of what that filter might look like this:

global_filter=[ "a|.*|","r|/dev/disk/by-id/id-of-unwanted-disk.*|" ]

The statements are encapsulated by square brackets. Statements are encapsulated by quotes, and are separated by a comma. Bars are used to open and close statements that contain slashes (like these do). A dot is put before an asterisk in that reject statement per regex standard to define a catchall wildcard. This line says, "Allow everything but this one drive, and all partitions that may come from this drive." This reject drive in question would be the LUN you don't want LVM to look at.

When you've saved this edit, test it out with "pvscan". If it went that way you wanted, your undesireable VG will no longer show. Do the same thing on the other host, and you have yourself a safe filter.

But seriously, isolate these LUNS using iSCSI targets if you can. It's much easier, and doesn't require configurations on both hosts to succeed.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.