Bug 406554

Summary: initrd created by mkinitrd does not boot
Product: [openSUSE] openSUSE 11.0 Reporter: Krister Lundmark <krille.lundmark>
Component: BootloaderAssignee: Alexander Graf <agraf>
Status: RESOLVED FIXED QA Contact: Jiri Srain <jsrain>
Severity: Major    
Priority: P5 - None CC: arun, forgotten_mbQyAD5r4K, heiko.schroeder, Ulrich.Windl, willems.luc
Version: Final   
Target Milestone: ---   
Hardware: x86   
OS: Other   
Whiteboard:
Found By: Customer Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---
Attachments: mkinitrd-mixed-lvm-system

Description Krister Lundmark 2008-07-04 17:32:22 UTC
When creating a new initrd using the command "mkinitrd", the resulting initrd file is unbootable.

The boot gets stuck in a loop when running the scripts on initrd, looking for the root partition. 

The output is:

!/boot/01-devfunctions.sh: multipath: command not found

So the only option is to boot in rescue mode, and reinstall the kernel (or use a backup of the original initrd file).

The same problem is recreatable both on default and pae kernel.

-----------
Info: 
-----------
- The system is on a laptop, single disk, lvm (not root), no multipath:

uname -r:
2.6.25.5-1.1-default

fdisk -l (root fs is on sda2):
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1          11       88326   de  Dell Utility
/dev/sda2   *          12         273     2104515   83  Linux
/dev/sda3             274       30401   242003160   8e  Linux LVM

lvdisplay | grep 'LV Name':
  LV Name                /dev/sys/home
  LV Name                /dev/sys/opt
  LV Name                /dev/sys/srv
  LV Name                /dev/sys/swap
  LV Name                /dev/sys/tmp
  LV Name                /dev/sys/u01
  LV Name                /dev/sys/usr
  LV Name                /dev/sys/var
Comment 1 Krister Lundmark 2008-07-05 08:05:17 UTC
I have made some more research: It seems that the problem is in the script 84-mount.. that calls 01-devfunctions.sh to search for the root device. Somehow it doesn't find the root device. I have tried to pass /dev/sda as "root" kernel parameter, but it still can't find the device (which btw is mountable from the boot environment when the script has stopped).

I have also extracted the default initrd file and the newly created, and compared the content. The default doesn't have any lvm related files, but the custom one does. The script files are identical in the two initrd files.
Comment 2 Krister Lundmark 2008-07-06 20:21:18 UTC
More research:
- I installed multipath packages (although they are not used);
multipath-tools-0.4.7-127.1
yast2-multipath-2.13.0-192.1

=> Still no boot

- I created a "monster initrd"

=> Boots

So the work-around is to create a monster initrd, but I still think this bug is major, since it can render a system unbootable...
Comment 3 Arun Persaud 2008-07-10 19:32:34 UTC
Got the same problem after latest kernel-pae update on openSUSE 11.0 two days ago. System is now unbootable with the error message

!/boot/01-devfunctions.sh: multipath: command not found

can mount the harddrive though as mentioned in comment #1. No idea on how to repair it, since both the rescue system and the normal kernel in grub fail to boot.

Comment 4 Arun Persaud 2008-07-11 06:42:17 UTC
managed to repair my system by running mkinitrd from the LiveCD. There is also a forum post at:

http://forums.opensuse.org/install-boot-login/389309-kernel-update-2.html#post1837090

Comment 5 luc willems 2008-07-12 09:10:47 UTC
same problem here , after installation and latest kernel update booting was not possible. had to recreate initrd manually like described in #4

my grub menu looks like
itle openSUSE 11.0 - 2.6.25.9-0.2 (default)
    root (hd0,4)
    kernel /vmlinuz-2.6.25.9-0.2-default root=/dev/disk/by-id/scsi-SATA_WDC_WD12
00BEVS-_WD-WXEY06757643-part6 resume=/dev/system/swap splash=silent  showopts vg
a=0x361
    initrd /initrd-2.6.25.9-0.2-default

Comment 6 Heiko Schröder 2008-07-12 10:09:02 UTC
I have exactly the same problem and want to provide a few additional observations:

The setup-lvm2.sh script checks if any of the blockdevices needed for the root filesystem or for resume are using LVM2. If yes, it sets the root_lvm2 flag and adds the volume groups to vg_roots.

The root_lvm2 flag will make the boot-lvm2.sh script check if the root filesystem is on LVM2. If yes, it sets vg_root the volume group. If vg_root OR vg_roots is set, it will then set root_major to the major device number of the device mapper. This will make the check_for_device functions check not only if the device for the root filesystem exists but additionally if it has the "correct" major number. In this case, thiw will oviously never happen as the root filesystem is not on LVM2.

I think, there are two possible solutions to the problem:

1. fix setup-lvm2.sh to set the root_lvm2 flag only if the root system is really on an LVM2 disk

2. let boot-lvm2.sh set root_major only if vg_root is set. This should only happen if the root filesystem is really on an LVM2 device

(Of course, solution 1 seems to be the cleaner one, though solution 2 should solve the problem equally well, if my observations are correct.)
Comment 7 Hannes Reinecke 2008-07-14 09:07:48 UTC
Can you check if this one fixes it?

diff --git a/scripts/boot-devfunctions.sh b/scripts/boot-devfunctions.sh
index 5cefc69..fab2291 100755
--- a/scripts/boot-devfunctions.sh
+++ b/scripts/boot-devfunctions.sh
@@ -63,7 +63,7 @@ check_for_device() {
                        echo " ok"
                        retval=0
                        break;
-                   else
+                   elif [ -x /sbin/multipath ] ; then
                        echo -n "!"
                        multipath -v0
                        wait_for_events
Comment 8 Heiko Schröder 2008-07-14 13:30:07 UTC
At least in my case: It can't as I don't have multipath installed at all.

And the problem is not that the device is not available (it's there and it has the name the scripts are looking for). The problem is that the scripts "think" that the root device is on a LVM2 device and thus search for the wrong major device id. The correct device is a standard ATA partition: /dev/disk/by-id/scsi-SATA_SAMSUNG_SP1654NS0GEJ1JL410834-part1 
(which points to /dev/sda1) and this is even show as "fallback". But as root_major is set to 253, a device with major 8 won't be used even if the name matches.
Comment 9 Hannes Reinecke 2008-07-14 14:07:26 UTC
Yes, I found that out myself.
Comment 10 Hannes Reinecke 2008-07-14 14:08:19 UTC
Created attachment 227621 [details]
mkinitrd-mixed-lvm-system

Patch to setup a mixed LVM system correctly.
Comment 11 Hannes Reinecke 2008-07-14 14:08:52 UTC
With the above patch I'm able to boot the system correctly. Will be included in the next mkinitrd update.
Comment 12 Ulrich Windl 2008-07-28 07:31:41 UTC
I don't want to beat dead horses, but when I updates my 11.0 DVD install system to the patches available on Friday, I ran into the problem with the pae kernel. The problem is nasty to repair, especially as the "repair" function of Yast fails to do so. I'd say this problem is "critical". The system with the problem had everything in one partition and multipath-tools installed. Still the message "multipath not found". My fix was to install the second to latest kernel via rpm from the rescue system. When updating another system that had a separate /boot partition, the problem did not appear.
Comment 13 Arun Persaud 2008-07-29 00:13:39 UTC
same here... last update messed up my computer again. Should this bug be reopened? or did the patch just not make it into that update yet?
Comment 14 luc willems 2008-09-12 18:50:19 UTC
hello guys

latest kernel update 
Linux lieve 2.6.25.16-0.1-pae #1 SMP 2008-08-21 00:34:25 +0200 i686 i686 i386 GNU/Linux

broke my booting "again" . this time i could get it fixed by recreating 
the initrd file :-(
during booting I can manually mount my root partition and exit the shell
after that , boot continous.
only this is a little bit hard to explain to my wife ;-)

need to test the patch mentioned above but please provide a fix.
it doesn't feel good if kernel updates break booting.



Comment 15 luc willems 2008-09-12 18:53:13 UTC
Some extra info :
i also have mixed environment
fdisk -l

Disk /dev/sda: 120.0 GB, 120034123776 bytes
255 heads, 63 sectors/track, 14593 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xc311042a

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1        3657    29366820    7  HPFS/NTFS
/dev/sda2            3657       14593    87851421    f  W95 Ext'd (LBA)
/dev/sda5            3657        3689      265009+  83  Linux
/dev/sda6            3690        7606    31463271   83  Linux
/dev/sda7            7607       14593    56123046   8e  Linux LVM

Disk /dev/dm-0: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

Disk /dev/dm-0 doesn't contain a valid partition table

Disk /dev/dm-1: 536 MB, 536870912 bytes
255 heads, 63 sectors/track, 65 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

Disk /dev/dm-1 doesn't contain a valid partition table


lvdisplay |grep "LV Name"
  LV Name                /dev/system/hoe
  LV Name                /dev/system/swap
Comment 16 luc willems 2008-09-12 19:12:40 UTC
ok, used patch from comment #10 with src rpm mkinitrd-2.2-19.1.src.rpm
and rebuild mkinitrd , installed it , run mkinitrd and now system
boot's correctly

so for me the patch seems to work :-)


Comment 18 Alexander Graf 2008-12-04 15:53:15 UTC
Is anyone still having issues with this? If not, I'll close the bug again.
Comment 19 Arun Persaud 2008-12-04 19:15:30 UTC
Everything seems to be ok at my end... updated the kernel once or twice (I think) and no problems.
Comment 20 Heiko Schröder 2008-12-04 19:30:13 UTC
The last kernel update worked without a problem, so it's OK for me, too.
Comment 21 Alexander Graf 2008-12-06 09:57:33 UTC
Great, so the patch fixed it. Thanks for reporting this and helping out everyone!