Bug 483136 - boot fails with cmdline root=LABEL= syntax (could not find /dev/root)
Summary: boot fails with cmdline root=LABEL= syntax (could not find /dev/root)
Status: RESOLVED FIXED
: 512695 548642 (view as bug list)
Alias: None
Product: openSUSE 11.3
Classification: openSUSE
Component: Kernel (show other bugs)
Version: Milestone 2
Hardware: i586 Other
: P3 - Medium : Normal (vote)
Target Milestone: ---
Assignee: Michal Marek
QA Contact: E-mail List
URL:
Whiteboard: maint:released:11.2:31186
Keywords:
Depends on:
Blocks:
 
Reported: 2009-03-07 03:52 UTC by Felix Miata
Modified: 2016-04-15 09:33 UTC (History)
5 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments
tgz of current & prior /var/log messages & boot.msg (44.58 KB, application/x-gzip)
2009-03-07 03:52 UTC, Felix Miata
Details
/lib/mkinitrd/scripts/setup-storage.sh with root=LABEL= syntax enabled (10.89 KB, text/plain)
2009-10-27 15:28 UTC, Milan Vančura
Details
patch as suggested by Kay's comment #18. (620 bytes, patch)
2010-01-15 16:10 UTC, Lars Müller
Details | Diff
Add patch fitting to the gitorious.org/opensuse/mkinitrd requirements. (1.34 KB, patch)
2010-01-15 17:04 UTC, Lars Müller
Details | Diff
/root/mkinitrd-debug (106.25 KB, text/plain)
2010-02-22 14:00 UTC, Felix Miata
Details
/root/mkinitrd-debug (108.05 KB, text/plain)
2010-02-22 14:16 UTC, Felix Miata
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Felix Miata 2009-03-07 03:52:21 UTC
Created attachment 277819 [details]
tgz of current & prior /var/log messages & boot.msg

User-Agent:       Mozilla/5.0 (OS/2; U; Warp 4.5; en-US; rv:1.8.1.19) Gecko/20081212 SeaMonkey/1.1.14 (PmW)

Current kernel is 2.6.29-rc6-git1-2-pae, but this has been going on with several kernel upgrades since before Christmas. Boot halts with "waiting for device /dev/root to appear ... could not find /dev/root ... want to fall back to /dev/dev/disk/by-id/...part10". On answering y, boot proceeds normally. Previously working cmdline
kernel /boot/vmlinuz showopts root=LABEL=S16A-10factory
causes above message. If I replace root=LABEL=S16A-10factory with root=/dev/sda10 then boot proceeds normally.

This is my bug 397816 P4/ICH4 (Dell GX260) system, which has been on ata_piix and dm_linear for the 11.1 Factory. I've tried my best to eradicate dm_linear support for the / (only) HD in 11.2 Factory, as described in http://lists.opensuse.org/opensuse-factory/2009-02/msg00335.html thread, but this problem has survived my attempt to (possibly successfully) convert.

Reproducible: Always

Steps to Reproduce:
1.Try to boot using root=LABEL= cmdline syntax

Actual Results:  
Boot halts

Expected Results:  
Boot proceeds normally

Looks a lot like bug 300550 .

When I boot a very old 2.6.24 11.0 Factory kernel and initrd with piix /, boot proceeds normally, with / mounted on /dev/hda10.
Comment 1 Josef Reidinger 2009-03-16 12:07:02 UTC
Hi, root=LABEL=something is bad syntax, correct is root=/dev/disk/by-label/something ... I cannot reproduce it if I install 11.1 and mount by label...could you please specify what you do? or please attach yast logs, maybe it still have logs about this change. Thanks
Comment 2 Felix Miata 2009-03-16 14:51:23 UTC
(In reply to comment #1)
> Hi, root=LABEL=something is bad syntax

If this is "bad syntax", it is a recent regression. It should remain an alternate syntax. During the late stages of 11.1 development it continued to work as it did in 11.0, 10.3, 10.2 & probably before then. I have been using this syntax for booting all my Linux distros (*buntu, Mandriva, Fedora, openSUSE) for two years or more:

00srv:~ $ cat /etc/SuSE-release
openSUSE 11.0 (i586)
VERSION = 11.0
00srv:~ $ uname -r
2.6.27.7-10-pae
00srv:~ $ cat /proc/cmdline
root=LABEL=08-suse110 noresume splash=off vga=0x305 3
00srv:~ $ tune2fs -l /dev/sda8| grep ume
Filesystem volume name: 08-suse110

FWIW, some months ago I stopped booting from the Grub installed by the selected distro. I install Grub to the / partition, but but boot from a primary partition that does not bet mounted to /boot. The menu.lst on the real boot partition., which by convention I mount to /disks/boot, I manage myself. Until now, everything has been booting using root=LABEL= syntax.

> correct is
> root=/dev/disk/by-label/something ... 

That may be what the installer does now, but at least as recently as November's Factory YaST was still using root=LABEL= syntax here. Long strings may be fine for GUI puter applets to manage, but they're not so good for normal human brains or for fitting on a single viewport-length line while editing a config file.

> I cannot reproduce it if I install 11.1
> and mount by label...could you please specify what you do?

I've done no fresh installs in many months, waiting for a post-2.6.27 installation kernel so that I can install on libata onto whichever partition up to #63 I please. Other than specifying to mount by label to my pre-existing and pre-labeled partitions during installation, I don't remember doing anything more than written above that deviates from defaults.

>or please attach
> yast logs, maybe it still have logs about this change. Thanks

Most of my updating gets done via cmdline zypper. If you think any of the following will help, specify, and I will attach:
    4943 Jun  5  2008 config_diff_2008_06_05.log
     632 Jun  5  2008 y2logmkinitrd
     760 Jun  5  2008 y2log_bootloader
       0 Jun  5  2008 y2logMount
     977 Jun  5  2008 volume_info
    6124 Jun  5  2008 macro_inst_initial.ycp
     974 Jun  5  2008 disk_sda
    4256 Jun  5  2008 y2start.log
 6926800 Jun  5  2008 y2log-1
    6967 Sep 30 16:23 config_diff_2008_09_30.log
    2040 Oct  2 18:14 config_diff_2008_10_02.log
  328761 Oct  2 18:25 y2logRPM
    1077 Nov 30 00:38 config_diff_2008_11_30.log
 5033281 Feb 26 13:19 y2log
    4039 Feb 26 15:14 config_diff_2009_02_26.log
    2962 Mar  5 14:51 config_diff_2009_03_05.log
   91501 Mar  6 01:14 mkinitrd.log
15880749 Mar  6 01:16 perl-BL-standalone-log
Comment 3 Josef Reidinger 2009-03-16 15:27:11 UTC
OK, then it looks like problem of kernel (and not perl-Bootloader). Reassign it to kernel guys.
Comment 4 Jeff Mahoney 2009-05-12 18:24:53 UTC
root=LABEL=<label> should still work, but it's not a kernel issue. That's handled in the initrd.
Comment 5 Hannes Reinecke 2009-06-30 10:50:10 UTC
Milan, could you have a look here?
Comment 6 L. A. Walsh 2009-09-23 00:06:27 UTC
*** Bug 512695 has been marked as a duplicate of this bug. ***
Comment 7 Felix Miata 2009-10-12 13:07:41 UTC
Ping.
Comment 8 Per Jessen 2009-10-13 19:04:13 UTC
Just FYI - have just installed 11.2M8, and I can confirm Felix' findings that booting with root=LABEL=<something> on the cmdline does not work.
Comment 9 Milan Vančura 2009-10-26 16:27:55 UTC
*** Bug 548642 has been marked as a duplicate of this bug. ***
Comment 10 Milan Vančura 2009-10-27 15:28:17 UTC
Created attachment 324348 [details]
/lib/mkinitrd/scripts/setup-storage.sh with root=LABEL= syntax enabled

The problem is in /lib/mkinitrd/scripts/setup-storage.sh
Try the attached one.
Comment 11 Felix Miata 2009-10-27 18:32:27 UTC
I tried comment 10 attachment with my comment 0 system. It had no apparent impact with either 2.6.31-10-pae or 2.6.31-10-desktop. I didn't try booting older kernels.
Comment 12 Harald Koenig 2009-10-29 09:08:05 UTC
the following patch works for me (but is ugly;)

--- /lib/mkinitrd/scripts/boot-storage.sh.orig      2009-08-11 11:59:50.000000000 +0200
+++ /lib/mkinitrd/scripts/boot-storage.sh   2009-10-29 10:07:17.000000000 +0100
@@ -49,6 +49,7 @@
         LABEL=*)
             label=${rootdev#LABEL=}
             echo "ENV{ID_FS_USAGE}==\"filesystem|other\", ENV{ID_FS_LABEL_SAFE}==\"$label\", SYMLINK+=\"root\"" > /etc/udev/rules.d/99-mkinitrd-label.rules
+           [ -e /dev/root ] || ln -s /dev/disk/by-label/$label /dev/root
             rootdev=/dev/root
             ;;
         UUID=*)
Comment 13 Felix Miata 2009-11-01 22:14:33 UTC
I just did a 'zypper dup' from factory-snapshot on a 11.1.rc2 ICH4 system running piix rather than ata_piix, which gave it both default and pae 2.6.31.5 kernels. Neither root=LABEL= nor root=/dev/hdX## syntax nor root=/dev/sdX## syntax work with either kernel. root=/dev/disk/by-path/pci* syntax does work, as does just as well absolutely no root= parameter whatsoever.

If I exit the Grub menu all the way to the Grub shell instead of choosing from menu.lst selections, then /dev/sdX## syntax works, but root=LABEL= syntax does not, while in oS11.1 final both work.
Comment 14 Lars Müller 2009-12-12 23:34:10 UTC
Created openSUSE Build Service request ID 26331 to merge the suggested fix from comment #12.

If it works for Factory I suggest to merge it to openSUSE 11.2 as well.
Comment 15 Lars Müller 2009-12-12 23:53:01 UTC
In http://lists.opensuse.org/opensuse/2009-12/msg00804.html Felix wrote:

### Felix begin
On 2009/12/09 02:04 (GMT-0500) Bob S composed:

> Any further instruction/guidance is certainly appreciated.

A reread of https://bugzilla.novell.com/show_bug.cgi?id=483136#c13 gave me
another idea to try: remove the whole root= string from the Grub kernel line
and see what happens.
### Felix end

If that works then we have a much simpler solution to address this issue.
Comment 16 Harald Koenig 2009-12-13 16:16:17 UTC
> If that works then we have a much simpler solution to address this issue.

that's no solution if I have multiple root partitions which I'd like to distingish/select/boot (and I do have more than one rootfs on my notebook).
even if yast won't create those grub entries, they are valid and they worked in older versions, so there is no reason to break them now (esp. since it's not hard to fix...)
Comment 17 Lars Müller 2009-12-15 17:41:35 UTC
Filed a new submit request (26621) to be non conflicting with the recent mkinitrd change.
Comment 18 Kay Sievers 2009-12-25 12:27:13 UTC
  + [ -e /dev/root ] || ln -s /dev/disk/by-label/$label /dev/root

We do not want to do nasty hacks like this. :) The real issue should be fixed instead of creating hacks like this. If nothing else is broken, I guess ID_FS_LABEL_SAFE should just be ID_FS_LABEL_ENC.
Comment 19 Lars Müller 2010-01-15 16:10:00 UTC
Created attachment 336838 [details]
patch as suggested by Kay's comment #18.

@Felix and anyone else using this feature: please test if this works for you too.
Comment 20 Lars Müller 2010-01-15 16:34:51 UTC
Created request id 29761 to merge the tested patch from comment 18 + 19.
Comment 21 Lars Müller 2010-01-15 17:04:22 UTC
Created attachment 336860 [details]
Add patch fitting to the gitorious.org/opensuse/mkinitrd requirements.
Comment 22 Lars Müller 2010-01-15 17:06:23 UTC
Comment on attachment 336838 [details]
patch as suggested by Kay's comment #18.

Obsoleted by attachment 336860 [details]
Comment 23 Lars Müller 2010-01-15 17:10:35 UTC
Comment on attachment 336838 [details]
patch as suggested by Kay's comment #18.

This patch is still valid for straight testing in a system.
Comment 24 Lars Müller 2010-01-15 17:11:53 UTC
Revoked the openSUSE Build Service request from comment #20.
Comment 25 Felix Miata 2010-01-15 21:56:19 UTC
Tried on one ICH4 machine on both 11.2 and Factory. Worked on both. :-)

(In reply to comment #15)
> remove the whole root= string from the Grub kernel line
> and see what happens.
> ### Felix end

> If that works then we have a much simpler solution to address this issue.

This is still valid. Totally removing root=whatever from the Grub kernel line does not seem to impose any obstacle to getting current Factory up and running.
Comment 26 Harald Koenig 2010-01-16 08:05:30 UTC
(In reply to comment #15)
> A reread of https://bugzilla.novell.com/show_bug.cgi?id=483136#c13 gave me
> another idea to try: remove the whole root= string from the Grub kernel line
> and see what happens.

I'm not sure that I get you right but completely removing root= is not always an option: I do have multiple root partitions on my notebook disk (11.1, 11.2, 32/64 bit, testing, maybe again a factory system at some time...) so I have to specif root=... to tell which fs to boot ?!?
Comment 27 Felix Miata 2010-01-16 09:14:02 UTC
I was suggesting something is wrong if boot succeeds in the absence of root=... on cmdline. I have few systems on which fewer than several root filesystems exist.
Comment 28 Michal Marek 2010-01-19 14:22:03 UTC
(In reply to comment #21)
> Created an attachment (id=336860) [details]
> Add patch fitting to the gitorious.org/opensuse/mkinitrd requirements.

Thanks, applied.
Comment 29 Felix Miata 2010-02-22 02:58:46 UTC
On a fresh M2 install on software RAID1, both root=LABEL=10suse113 and root=/dev/md3 fail. The fallbacks don't work either. The only way I've found to boot M2 so far is root=/dev/sda10.
Comment 30 Michal Marek 2010-02-22 10:32:12 UTC
Are there any error messages printed besides the 
Are you able to manually assemble the array in the emergency shell?
# mdadm --assemle /dev/md3 /dev/sda10 /dev/...
can you boot with linuxrc=debug and capture the mkinird output via a serial console?
I need some more data "it fails" :).
Comment 31 Felix Miata 2010-02-22 12:14:15 UTC
"It fails" means the same thing here as I wrote in comment 0. Quoting comment 0: "Waiting for device /dev/root to appear ... could not find /dev/root ... want to fall back to /dev/....". With root=LABEL=10suse113 on cmdline, it's /dev/md3 that it asks to fall back to, then after answering y, it's "Waiting for
device /dev/md3 to appear ... not found -- exiting to /bin/sh; sh: cannot set terminal process group (-1): Inappropriate ioctl for device; sh: no job control in this shell; $"

'$ mdadm --assemle /dev/md3 /dev/sda10 /dev/sdb10' appears to work, with 'mount -t ext3 /dev/md3 /' also working, but I don't know what to try next, as 'init 1' next fails.

If I boot with root=/dev/sda10 on cmdline I get what appears to be a normal boot result, with /dev/md10 on /. Should I add linuxrc=debug to that and then run mkinitrd? I don't know capture via serial console process. Will /boot/log/YaST2/mkinitrd.log provide the output you want instead?

I must have typed something wrong when I tried to boot by changing from root=LABEL=10suse113 to root=/dev/md10 in the gfx Grub edit box before writing comment 29. Now when I try root=/dev/md10 I get what seems to be a normal boot with no apparent waiting for /dev/root to appear. /dev/md[0-6] are all mounted where they belong according to mount output.
Comment 32 Michal Marek 2010-02-22 13:16:05 UTC
If you boot with linuxrc=debug, the initrd's /init will run under set -x, i.e. print all commands as they are executed. But you'd need a second computer and a serial cable to save the output, so let's try something different.

You say that root=/dev/md10 actually works, so at least it's not a problem with the md array assembly. Can you boot into the emergency shell again and do the following:
# mount /dev/md10 /root
# rm -f /root/tmp/mkinitrd-debug
# cat /etc/udev/rules.d/99-mkinitrd-label.rules >/root/tmp/mkinitrd-debug
# udevadm info --export-db >/root/tmp/mkinitrd-debug
# umount /root
# ln -s md10 /dev/root
# exit

Hopefully the system should continue booting after creating the symlink manually. Please attach the /tmp/mkinitrd-debug afterwards.
Comment 33 Michal Marek 2010-02-22 13:16:52 UTC
(In reply to comment #32)
> # cat /etc/udev/rules.d/99-mkinitrd-label.rules >/root/tmp/mkinitrd-debug
> # udevadm info --export-db >/root/tmp/mkinitrd-debug

Oops, please use '>>' here instead:
# udevadm info --export-db >>/root/tmp/mkinitrd-debug
Comment 34 Felix Miata 2010-02-22 14:00:23 UTC
Created attachment 343797 [details]
/root/mkinitrd-debug
Comment 35 Felix Miata 2010-02-22 14:01:27 UTC
(In reply to comment #32)
> Can you boot into the emergency shell again and do the
> following:
> # mount /dev/md10 /root

The choices here are /dev/md3, /dev/sda10 or /dev/sdb10, the latter two of which comprise /dev/md3.

$ mount -t ext3 /dev/md3 /root
mount: special device /dev/md3 does not exist

$ mdadm --assemble /dev/md3 /dev/sda10 /dev/sdb10
$ mount -t ext3 /dev/md3 /root
$ rm -f /root/tmp/mkinitrd-debug
$ cat /etc/udev/rules.d/99-mkinitrd-label.rules >/root/root/mkinitrd-debug
$ udevadm info --export-db >/root/root/mkinitrd-debug
$ umount /root
$ ln -s md3 /dev/root
ln: creating symbolic link `/dev/root': File exists
$exit

[apparently normal init]
Comment 36 Michal Marek 2010-02-22 14:06:59 UTC
Ah, I'm getting it. The problem here is that the md array is not assembled if you boot with root=LABEL=, because the md script does not know that an md root is needed. I think this scenario has never worked. BTW, once the array is assembled manually, the /dev/root symlink is correctly created, so this is a differenc bug than the original 11.2 report.
Comment 37 Felix Miata 2010-02-22 14:16:16 UTC
Created attachment 343802 [details]
/root/mkinitrd-debug

Looks like the first try I typed udevadm info --export-db >/root/root/mkinitrd-debug instead of udevadm info --export-db >>/root/root/mkinitrd-debug. :-(

I did /root/root/... instead of /root/tmp/... because normally /dev/md0 is mounted on /tmp.
Comment 38 Felix Miata 2010-02-22 14:28:20 UTC
(In reply to comment #36)
> Ah, I'm getting it. The problem here is that the md array is not assembled if
> you boot with root=LABEL=, because the md script does not know that an md root
> is needed. I think this scenario has never worked.

I think it has:
# cat /etc/SuSE-release
openSUSE 11.0 (i586)
VERSION = 11.0

# uname -r
2.6.27.37-0.3-pae

# cat /proc/cmdline
root=LABEL=08suse110 noresume splash=off vga=0x305 3

# mount | grep md
/dev/md1 on / type ext3 (rw,noatime,acl,user_xattr)
/dev/md0 on /tmp type ext3 (rw,noatime,acl,user_xattr)
/dev/md4 on /pub type ext3 (rw,noatime,acl,user_xattr)
/dev/md5 on /srv type ext3 (rw,noatime,acl,user_xattr)
/dev/md6 on /home type ext3 (rw,noatime,acl,user_xattr)
Comment 39 Felix Miata 2010-02-22 18:01:48 UTC
(In reply to comment #36)
> BTW, once the array is
> assembled manually, the /dev/root symlink is correctly created, so this is a
> differenc bug than the original 11.2 report.

This bug does seem fixed on a different/non-RAID fresh M2 install, where I'm booted with 'cat /proc/cmdline':

root=LABEL=H16A-22factory noresume splash=verbose vga=0x305 3
Comment 40 Swamp Workflow Management 2010-02-23 14:13:29 UTC
Update released for: mkinitrd, mkinitrd-debuginfo, mkinitrd-debugsource
Products:
openSUSE 11.2 (debug, i586, x86_64)
Comment 41 Felix Miata 2010-02-25 21:46:05 UTC
(In reply to comment #36)
> once the array is
> assembled manually, the /dev/root symlink is correctly created, so this is a
> differenc bug than the original 11.2 report.

Bug 583424 filed due to above comment.
Comment 42 Michal Marek 2010-03-09 10:40:44 UTC
OK, let's close this one then.
Comment 43 Felix Miata 2010-03-15 08:21:45 UTC
This was filed against 11.2. I just updated an 11.2 system to test this. It wasn't fixed. Then I ran mkinitrd, and it still isn't fixed.
Comment 44 Bernhard Wiedemann 2016-04-15 09:33:13 UTC
This is an autogenerated message for OBS integration:
This bug (483136) was mentioned in
https://build.opensuse.org/request/show/30010 Factory / mkinitrd