56

I currently have LVM on software RAID, but I'd like to ask you what you think it is better solution, maybe some pros and cons?

Edit: It is about software raid on lvm or lvm on software raid. I know than hardware raid is better if we are thinking about performance.

3
  • 10
    lvm on raid. raid on lvm is the devil's work. -- no reasoning.
    – Sirex
    Dec 31, 2010 at 13:12
  • well, hw raid is not necessary better that sw raid. It depends on several factors. If it not a high-end one it is usally worse than linux sw raid (aka mdadm). Regarding management, stability and performance.
    – cstamas
    Feb 24, 2011 at 1:34
  • @cstamas: I agree, because cheap ones are usually actually software raids.
    – Ency
    Feb 24, 2011 at 6:46

6 Answers 6

63

Your current setup is like this:

| / | /var | /usr | /home  |
 --------------------------
|       LVM Volume         |
 --------------------------
|       RAID Volume        |
 --------------------------
| Disk 1 | Disk 2 | Disk 3 | 

It's a much simpler setup with more flexibility. You can use all of the disks in the RAID volume and slice and dice them whatever way you like with LVM. The other way isn't even worth thinking about - it's ridiculously complicated and you lose the benefits of LVM at the filesystem level.

If you tried to RAID LVM volumes, you would be left with a normal device without any of the LVM volume benefits (e.g. growing filesystems etc.)

14

This is an old question, technology has advanced and the recommended setup is to use built-in RAID support of LVM (see here for setup https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Logical_Volume_Manager_Administration/raid_volumes.html ), especially if you are using SSD. Red Hat does not recommend the use of RAID 1/5/6/10 with SSD since mdadm will write the complete partition to ensure proper functioning of checksums. This can lead to faster degradation of the SSD, as stated here https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html-single/Storage_Administration_Guide/index.html#ssddeploy

4
  • 2
    USER irwinr suggested: Edited Feb 4th 2018: I am unable to find any reference to RedHat recommending LVM RAID over mdadm RAID nor any caution against using RAID with SSDs. In fact the #ssddeploy tag does not even appear in the HTML for the 2nd linked page. ---- Thanks irwinr, it looks like RedHat updated their documentation. Still mdadm syncs whole partition, while "smart" LVM does not, hence less unnecessary disk writes, hence longer life for your SSD. Jun 30, 2018 at 0:58
  • Just a thought: although you have the basic idea in a comment you might want to edit it into the answer maybe even with a disclaimer; that is to say that you should have a warning of some kind that says under what circumstance(s) you're suggesting your method.
    – Pryftan
    Dec 18, 2018 at 14:08
  • 1
    I'm guessing LVM is just using the mdam option --assume-clean behind the scene i.e. still no advantage of using LVM-RAID instead of mdadm (at leat not in that regard). If your're happy with no full resync when create array with mdadm just use that optino yourself. Today, 4 years later, resync the whole SSD disk make no difference in regards to wear-out as the endurance-levels on newer SSD are so high.
    – MrCalvin
    Oct 22, 2020 at 4:12
  • There is a huge benefit in having LV based RAID. I do have a LV with RAID1 for more hard to reproduce data and normal LVs for the rest, this gives me maximum flexibility for size vs. redundancy considerations.
    – wiebel
    Sep 20, 2022 at 10:21
4

Your current setup is fine. This is the recommended way to do it.

Raid deals with keeping the bits secure/redundant/fast/whatever and LVM helps you present them in a esasy to use way.

2

have hardware raid and you can have lvm on top - best combination.

1
  • 4
    Good enterprise solution, but an awful suggestion for homelab! You need a service contract with the hardware supplier or you're in deep trouble if your RAID hardware has a fault - you need to find compatible proprietary hardware to replace it. Mar 15, 2019 at 10:57
0

I am assuming you mean Hardware RAID with LVM on Top, vs. LVM and Software RAID on top of LVM. If so, I always advise to opt for hardware based RAID first. Software RAID is just that, while overhead is small, hardware RAID performance will be better 9 out of 10 times. Of course, the methodology will very much depend on your final goal. What are you trying to achieve (performance, protection, etc., etc.)

1
  • 2
    "lvm and software raid on top of LVM" -- Now that's an exotic setup !
    – Sirex
    Dec 31, 2010 at 13:50
-3

I think it makes sens to use RAID over LVM if you want to split your disks between a RAID 0 volume and a RAID 1 volume.

With this you can't realocate space between RAID0 and RAID1

| / | /var | /usr | /home  |
 --------------------------
|       LVM Volume 2       |
 --------------------------
|    RAID 0   |   Raid 1   |
 --------------------------
| Disk 1 | Disk 2 | Disk 3 |

With this you can

| / | /var | /usr | /home  |
 --------------------------
|       LVM Volume 2       |
 --------------------------
|    RAID 0   |   Raid 1   |
 --------------------------
|       LVM Volume 1       |
 --------------------------
| Disk 1 | Disk 2 | Disk 3 | 

the good thing is that you can also still move the LVM Volume 2 logical volumes between raid 0 et and raid 1 volume, by using pvmove command

the bad thing is that the setup is complex It would be better if LVM had a better integration of software raid features.

3
  • 7
    I don't know enough storage goodies to say why this is a bad idea, but it leaves me quite unsettled and a distinct feeling of dread.
    – Scott Pack
    Nov 7, 2012 at 16:15
  • 2
    If "RAID" here was RAID 1 / 5 / some other form of redundant RAID, you would not get any redundancy. If any disks under the LVM Volume 1 failed, you would lose both RAID volumes, as the LVM volume would be dead (OK, you could force it online still assuming that it wasn't striping, but a large amount of the data would now be gone, and the RAID ontop of the failing LVM volume probably won't be able to recover...). I don't know why you would do this, but don't.
    – BSchlinker
    Nov 7, 2015 at 0:16
  • 3
    Scott, it seems that your dread was justified. I used mdadm RAID over LVM2 and got myself into trouble - serverfault.com/questions/826479/… - let my experience be warning to others.
    – Ghostrider
    Jan 16, 2017 at 6:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .