0

BusyBox why u no vi?!

Summary:

Ubuntu 14.04 Server after upgrade first boot, only boots to BusyBox v1.21.1 initramfs with "gave up waiting for root device" due to missing /dev/mapper/xxx-root as unable to find/mount LVM VG due to incorrect exclusion filter in lvm.conf

Scenario:

  • only have the single kernel 3.11.0-17 (yes, no fallback!), both normal and recovery go to BusyBox
  • only ssh access, or IP KVM (system in remote CoLo in DC over 10,000 miles away)
  • no access to bootable media as IP KVM does not load virtual media
  • default initramfs configuration (attempt to use vi shows vi: not found)
  • LVM PV should be on something like /dev/sda5 (assume this as have many systems with same config and that is what the others are set as)

Detail:

100% sure issue is incorrect LVM filter (configuring OpenStack Cinder loopback device for LVM poorly), consequently is not scanning for LVM VG. Known working should be default in lvm.conf:

filter = [ "a/.*/" ]

However this is commented out and instead have in lvm.conf:

filter = [ "a/sda1/", "a/sdb/", "r/.*/"]

If I can make the change in lvm.conf by searching for incorrect string and replacing line with correct string in lvm.conf then I believe LVM VG will boot correctly.

Attempted Solution:

initramfs has no editing (vi, vim, nano, etc), only sed have tried testing sed regex, sadly this is incredibly difficult to crack for this regex noob. I have so far:

sed -i '/filter = [ "a/sda1/", "a/sdb/", "r/.*/"]/c\filter = [ "a/.*/" ]' /etc/lvm/lvm.conf

which makes no change to a test file. I have trouble comprehending the complexities of slashes contained within the sed search/replace string in conjunction with the sed command slash syntax, maybe someone can assist here? If I can crack the sed command regex I'm fairly sure the kernel should be able to boot into LVM.

Alternatively is there a force/manual mount I could use (from grub edit or initramfs), despite the incorrect lvm.conf filter, what is this?

As a fallback, I could go down the path of MaaS (Ubuntu's web based PXE/tftp image booting), which I have configured on another server in the DC, but have not yet tested, or used. I am happy to try to blast a new Ubuntu 14.04 Server onto the non-booting system using MaaS.

Correctly editing lvm.conf by cracking sed first in BusyBox is more desired. I'd like to understand this for future, or any other suggestions? Edit at grub, manual LVM VG boot, or any other suggestions that help are appreciated. Thank you.


Follow up edit:

(not sure on serverfault best practice here, whether I should answer my own question or make this edit, anyway..)

A big thank you to mvillar for the sed guidance. OK, here is the final synatax and steps that saved my bacon from an unbootable system (only to BusyBox) due to an error in lvm.conf filter configuration. I first used the following sed command to double check for correct syntax before and after change to '# print only lines which match regular expression (emulates "grep")':

sed -n '/loop/p' /etc/lvm/lvm.conf

Source: http://www.catonmat.net/blog/wp-content/uploads/2008/09/sed1line.txt

So I discovered the actual incorrect line I had in lvm.conf (differing from what I originally stated):

filter = [ "a/sdc5*/","a/loop2*/", "r/.*/" ]

Note: I believe the cause of the booting error was incorrect syntax in the original lvm.conf. I wanted: filter to accept sda5 and loop2 and reject all others. I was incorrectly stating: sdc5 and possibly missing space after comma

sed regex that worked to replace incorrect syntax with 'accept all' filter in lvm.conf:

sed -i "s/filter \= \[ \"a\/sdc5\*\/\",\"a\/loop2\*\/\", \"r\/\.\*\/\" \]/filter = [ \"a\/.\*\/\" ]/" /etc/lvm/lvm.conf

Lessons learned: + escape not needed for spaces, needed for literal string forward slash, asterix and quotes + default Ubuntu 14.04 does NOT include vi or tail, despite them both listed as BusyBox commands in upstream, something awry here with Ubuntu BusyBox default build. I will in future check that all my systems include AT LEAST vi and tail in BusyBox builds! + sed is at first complicated to understand, but a really useful and powerful tool

Steps to view LVM status, recreate LVM vg and lv and boot from BusyBox after sed and checking:

(initramfs) lvm
lvm> pvdisplay
lvm> vgdisplay
lvm> lvdisplay
lvm> vgchange -a y
lvm> exit
(initramfs) exit

Source: http://ubuntuforums.org/showthread.php?t=1898901

Ubuntu then continued normal boot with active root lv and went straight to login. Awesome!

1

According to the docs, busybox 1.12.1 has vi.

Anyway, here's your sed regex:

sed -i "s/filter\ \=\ \[\ \"a\/loop2\/\",\ \"r\/\.\*\/\"\ \]/filter = [ \"a\/.*\/\" ]/" /etc/lvm/lvm.conf 
  • Yes, I noticed that in the docs and double checked, definately vi is not listed in commands 'help' inside BusyBox and command not found when using. Thank you for your sed regex, I tried but unfortunately still unable to get it to change the line in my test. Unsure what is not picking up correctly... – Marten Hauville Jul 9 '14 at 6:35
  • Yes, I had put "a/.*/" instead of "r/.*/" in the first part of the sed. Try the updated answer. – mvillar Jul 9 '14 at 6:38
  • Had another check and the syntax is different to what I said earlier (sorry), it is actually: 'code' filter = [ "a/loop2*/", "r/.*/" ] – Marten Hauville Jul 9 '14 at 6:42
  • So including your edit, I think it should be: 'code' sed -i "s/filter\ \=\ [\ \"a\/loop2\/\",\ "r\/\.*\/\"\ ]/filter = [ \"a\/.*\/\" ]/" /etc/lvm/lvm.conf – Marten Hauville Jul 9 '14 at 6:47
  • You have to escape the " and * in the r part. – mvillar Jul 9 '14 at 6:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.