0

We have a dedicated server with software RAID1 and one of the disk failed recently. The disk was replaced but after rebuilding the array and rebooting the server freezes with a Kernel Panic message

No filesystem could mount root, tried: reiserfs ext3 ext2 cramfs msdos vfat iso9660 romfs fuseblk xfs
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(9,1)

The filesystem on both disks is ext4.

It seems the kernel can't load ext4 support.

Is there any way to add ext4 support or do I need to recompile a new kernel again ?

Interesting point that before disk replacement all was fine.

The kernel is a stock kernel bzImage-2.6.34.6-xxxx-grs-ipv6-64 from our provider OVH

Here is the content of my lilo.conf file cat /etc/lilo.conf

lba32
boot=/dev/md1
raid-extra-boot=mbr-only
prompt
timeout=50

# Enable large memory mode.

large-memory
image=/boot/bzImage-2.6.34.6-xxxx-grs-ipv6-64
    label="Linux"
    root=/dev/md1
    read-only
1
  • Can we have the whole of your grub.conf entry, relating to the current kernel? In particular, are you booting with an initrd image?
    – MadHatter
    Mar 16, 2011 at 7:36

5 Answers 5

2

I have managed to solve the issue. After the disk replacement the new disk didn't have boot code installed on it. So I have reinstalled lilo and then rebooted again. lilo -H -v Now everything works fine.

1
  • 1
    you can mark this as the solution by clicking the tick next to this answer. Marking this as the solution shows anyone else on the internet who finds this question that this solved your problem. Mar 17, 2011 at 8:36
0

Filesystem driver can also be loaded from the initrd. Make sure grub (or whatever bootloader you use) loads the right initrd. Maybe you're loading an old kernel.

If you can boot with a rescue system you can rebuild the initrd.

0
0

Atleast in grub2 you can specify to load a few modules before booting..see http://grub.enbug.org/FranklinPiat/grub_modules.manpage

Have used it for lvm. But if you had compiled the kernel from source, did you compile ext4 support at all?? the grub solution will work only if you had compiled ext4 as a loadable module. if not bad luck..gotta recompile...

0

If I where you I would first try to boot the server with a live cd and see if the raid is still ok. If the kernel could read the ext4 before the disk failure then it should be able to read it now so I think there might be something else that is wrong like the raid couldn't be autostarted, filesystem corrupt or something like that.

1
  • Sorry but i do not have access to the physical console of the server. I checked the raid array and it rebuilded successfully. Also at boot time it seems the kernel sees the disks as IDE but they are SATA
    – cioby23
    Mar 16, 2011 at 8:33
0

Interestingly, block device 9,1 is indeed /dev/md1:

brw-r-----  1 root disk    9,     1 Feb 15 16:13 md1

Is it possible that the RAID devices have become renumbered as a result of the hardware failure? You say you don't have physical kernel access so can't boot of removable media (eg a live CD), but you have posted a kernel panic message, and you propose to rebuild the kernel, which will be difficult to do if you can't get the machine up; so I am slightly confused about what you can and can't do, which makes it hard to give you options.

The issue probably isn't ext4, because it booted from that root FS before.

My inclination at the moment is to suspect that the current kernel doesn't have MD support built-in (not a loadable module, but compiled in). The message you print is eactly what I'd expect to see if you were to boot off a non-md-aware kernel - but you don't say that you changed that.

So I have to ask, firstly: what have you changed since the last time the box rebooted successfully, apart from a hard drive? You do mention "recompile a new kernel again", which makes me wonder if you've been changing the kernel a lot, recently. Is there any possibility that you've left MD support out of the current one? Are you in a position to give us the "Multi-device support (RAID and LVM)" section of your current kernel's config file?

2
  • i can enter remotely into a rescue mode and access this partition /dev/md1. To be more clear there are 2 disks in RAID 1 as /dev/sda which was replaced and /dev/sdb. After the disk was changed i have duplicated the partition table from /dev/sdb to /dev/sda and then rebuild the RAID 1 array using mdadm. After the rebuild was completed I have restarted the server from rescue mode and receive the kernel panic messages. This is the guide I have followed help.ovh.co.uk/RaidSoft
    – cioby23
    Mar 16, 2011 at 10:18
  • The fact that you can boot into rescue mode doesn't tell us that your current kernel supports MD RAID, so I repeat my question.
    – MadHatter
    Mar 16, 2011 at 14:53

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .