6

A server set up with Debian 6.0/squeeze. During the squeeze installation, I configured the two 500GB SATA disks (/dev/sda and /dev/sdb) as a RAID1 (managed with mdadm). The RAID keeps a 500 GB LVM volume group (vg0). In the volume group, there's a single logical volume (lv0). vg0-lv0 is formatted with extfs3 and mounted as root partition (no dedicated /boot partition). The system boots using GRUB2.

In normal use, the systems boots fine.

Also, when I tried and removed the second SATA drive (/dev/sdb) after a shutdown, the system came up without problem, and after reconnecting the drive, I was able to --re-add /dev/sdb1 to the RAID array.

But: After removing the first SATA drive (/dev/sda), the system won't boot any more! A GRUB welcome message shows up for a second, then the system reboots.

I tried to install GRUB2 manually on /dev/sdb ("grub-install /dev/sdb"), but that doesn't help.

Appearently squeeze fails to set up GRUB2 to launch from the second disk when the first disk is removed, which seems to be quite an essential feature when running this kind of Software RAID1, isn't it?

At the moment, I'm lost whether this is a problem with GRUB2, with LVM or with the RAID setup. Any hints?

4

You need to install GRUB to the MBR of both drives, and you need to do it in a way that GRUB considers each disk to be the first disk in the system.

GRUB uses its own enumeration for disks, which is abstracted from what the Linux kernel presents. You can change which device it thinks is the first disk (hd0), by using a "device" line in the grub shell, like so:

device (hd0) /dev/sdb

This tells grub that, for all subsequent commands, treat /dev/sdb as the disk hd0. From here you can complete the installation manually:

device (hd0) /dev/sdb
root (hd0,0)
setup (hd0)

This sets up GRUB on the first partition of the disk it considers to be hd0, which you've just set as /dev/sdb.

I do the same for both /dev/sda and /dev/sdb, just to be sure.

Edited to add: I always found the Gentoo Wiki handy, until I did this often enough to commit it to memory.

  • 1
    You're talking GRUB1. GRUB2 doesn't have a a setup command in the shell. – flight Mar 9 '11 at 17:10
  • You're probably right there. Carry on :) – Daniel Lawson Mar 9 '11 at 20:30
  • A bit more explanation here would help. Is the implication that on boot without sda, BIOS presents the second drive as ata0 while GRUB legacy tries to load menu.lst etc from hd1? Is there any equivalent caveat for grub2 (now confusingly just referred to as grub)? – Cedric Knight Aug 23 '17 at 6:32
2

Have you considered installing a third drive to serve as just the boot drive? I have seen problems too with raid 1 lvm setups (on CentOS) not being able to boot the second drive. I think the problem stems from grub not being able to handle native lvm partitions, although I'm not entirely sure.

Anyway, that's my answer: install a third small drive solely for the purpose of booting the system. Heck, I bet you could even get clever and do that with some sort of little flash or ssd device.

  • My current solution is to boot from a USB stick with GRUB2 on it (and a /boot filesystem, which is not exactly necessary, I think). – flight Mar 9 '11 at 17:13
  • 1
    Still I refrain from accepting this answer, since this ought to work without a third drive. From what I can tell, this has to be a bug in GRUB2 (in Debian Squeeze). – flight Mar 9 '11 at 17:14
  • Sure, that's a reasonable assumption. I just wanted to point out that I've seen weird lvm/raid/grub issues before, and solved it via a third drive via beating my head against weird annoying boot-time bugs. – Phil Hollenback Mar 9 '11 at 18:02
1

Grub should be able to recognize RAID1 setups and install to all slave disks when told to install to the MD device.

  • 1
    That's what I thought as well ;-), and yes, the Debconf frontend to grub-pc suggested an install in /dev/sda as well as /dev/sdb (and /dev/dm-0, where it failed to install subsequently). Still, it wouldn't boot with the second disk only. – flight Mar 9 '11 at 17:12
  • I dimly remember that one had to point it at the MD device rather than at the components, but I may be confusing that with LILO here. – Simon Richter Mar 9 '11 at 19:00
  • This appears to be how grub-install works with GRUB 1.99 and 2.02. In whatever way sda+sdb RAID1 holds your boot partition, the core is likely to be referenced by UUID (check my linked question to see if it is). So if you grub-install /dev/sda; grub-install /dev/sdb, it doesn't matter if you remove one of those drives: so long as the BIOS can load MBR from one of them, it will find the RAID UUID and LV by searching. – Cedric Knight Aug 27 '17 at 19:05
0

Indeed it should work. This appears to be how grub-install works with GRUB 1.99 and 2.02.

In whatever way sda+sdb RAID1 holds your boot partition, the core is likely to be referenced by UUID. Check my linked question to see if it is. In other words if grub-install --debug shows something like --prefix=(md0)/boot/grub/ you might get a problem if another RAID array is found first, which would probably give the grub rescue> prompt, rather than the crash observed here. If it uses --prefix=(mduuid/, it should find it.

So if you grub-install /dev/sda; grub-install /dev/sdb, it doesn't matter if you remove one of those drives: so long as the BIOS can load MBR from one of them, it will find the RAID UUID and LV by searching. The MBR, however, is not mirrored. So point the installer at all components in turn.

All that's in theory....


My interest in this ancient question is that the Welcome to GRUB! banner was shown and then the server rebooted, because I have similar symptoms, possibly caused by the BIOS being unable to read a 4K/sector drive. I don't know if the questioner ever found a solution.

The logic here is that the welcome message is found in kernel.img, so at least part of the core must be loading. However according to the (possibly outdated) manual, the second sector read loads all the rest of the image into memory using a blocklist. If the blocklist is damaged (or perhaps the LBA offset is computed wrongly for sector size or other reasons), then a crash, reboot or hang might occur.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.